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THE DISTRIBUTION OF MATRICES RESULTING FROM 
NEWTON'S IDENTITIES IN A FIELD OF CHARACTERISTIC TWO * 


E. R. Berlekamp 

Introduction: In order to decode a binary group code, one 

considers the set of all error patterns which have the same syndrome 
(pattern of parity check failures) as the received syndrome. From 
that set one chooses an error pattern which has the fewest errors. A solution 
to this problem is a set of err or -location numbers {p^}. Bose-Chaudhuri 
(1961a, b) have suggested that, if the block length, N, is one less than a 
power of 2, then the positions of the code (and hence the error -location 
numbers) may be taken as the nonzero elements of a finite field, 

GF(N + 1) = GF(2 ). Bose-Chaudhuri have demonstrated methods of 
choosing a parity check matrix for the code so that the syndrome of the 
received sequence may be interpreted as the first e odd power-sum 
symmetric functions, S. =X ^ * j = 1, 3, 2e - 1. The decoding 

problem for Bose-Chaudhuri codes consists of finding the {|3. }, given 
the S., j = 1, 3, .... 2e - 1. One form of a solution is the polynomial 

m 

a(x) = I | (x - (3. ) = T* or .x 1 . Chien (1964) has recently demonstrated an 
i i=0 1 

elegant, easily instrumented method for finding all the roots in GF(2 k ) 
of any polynomial (r (x), but this procedure still requires one to evaluate 
several determinants of matrices relating the <r's to the S's. Massey 
(1965) has suggested a method to reduce this work to the evaluation of a 
single determinant of the type we consider here. 

The relations between the cr's and the S's are given by Newton's 
Identities. (Peterson, p. 176): 


The research herein was supported by the Air Force Office of Scientific 
Research under Grant AF-AFOSR-639-64. 
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S 2 - V, + 2t 2 



0 

(2) 

S 3 - SjjTj + Sj, 2 

- 3(r 3 

sr 

0 

(3) 

S 4 “ S 3 cr l + S ^2 

- + 

4°4 = 
0 

0 

(4) 

• 

• 

In a field of characteristic 

• 

two, 

we have 

• 

additional equations. 


o 

S = S for all i. Every even numbered Newton's identity is equivalent 
2i 1 

to one of these equations. When the dependent equations are omitted and 
the remaining equations are converted into matrix form, we have 



It is convenient to consider S Q = «r Q = 1; S = cr . = 0 for all i > 0. 
Then the above equations become 

J S 2i-j-l°j = 0 for i = l, 2, e. (8) 

The decoding procedure of Peterson (1962) first finds a polynomial 
cr(x) of lowest degree which satisfies equations (8), if there is any such 
polynomial of degree m not more than e. If this polynomial has m 
distinct roots in GF(2 ), then these roots are the error positions. Notice 
that if this polynomial has m roots in GF( 2 k ), then they are distinct. 

For if o-(x) = (x - p Q ) <Tq(x), then a Q (x) is a polynomial of degree (m - 2), 
and the coefficients of (Tq(x) also satisfy (8). In general, however, the 
solution of (8) with the least degree may have nonlinear factors which are 

k 

irreducible over GF(2 ). 

Bose-Chaudhuri and Peterson have shown that Peterson's decoding 

procedure will correct any error pattern containing no more than e 

errors, and no others. There are ^ f^^such correctable patterns. 

i= 0 ' 1 ' 

But the total number of correctable error patterns is equal to the total 
number of syndromes. Except for the trivial (Hamming) case when e = 1, 
the number of correctable error patterns is much larger than J" 

i=0 ' 1 ' 

typically (N + 1) . The additional correctable error patterns have more 

than e errors, and each of them is less likely than any of the ^ 

i=0 ' 1 ' 

patterns which Peterson's algorithm can correct. But there are so many 
correctable error patterns with more than e errors that the overall error 
probability of the code would be substantially reduced by any improvements 
which enabled the decoder to tackle these cases. 

It is for this reason that we begin here an investigation of the 
properties of equations (8) over the entire ensemble of possible matrices 
corresponding to the (N + l) e possible values of S„ S„ ..., S_ .. We 
introduce an invertible transformation of equations (8) which reduces the 
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enumeration problem to one which has previously been solved by Daykin 
(I960) and Berlekamp (1963) independently. Since this transformation 
reduces the size of the matrix to be inverted by a factor of two, it might 
well prove useful in the construction of decoding circuits for this problem. 
We do not examine this possibility here. 

Triangular transformations: We now introduce an arbitrary 

triangular transformation of the equations (8). Let 

T k =X A j S k_ 2 j’ wh ere A Q = 1; A_. = 0 for i > 0. (9) 


Then if we add to each equation (8) times the previous equa- 
tions, A 2 times the equation before that, we transform (7) into 



or equivalently 


°2e-l 


X T 2i-j-l°j = 0 for 1 = 1 ‘ 2 » •••» e * 

Notice that this transformation is readily invertible: 


Sk ' T * + |i A A-2r 


(ID 


( 12 ) 






% 
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A 


0 


We now choose the A 1 s in such a way that T^. = 0, for all i. 
This is readily accomplished by setting 




or 


i-1 

a. = y a.s 9/ . ... 

1 jto 1 


(13) 


Given the initial condition that = 1, equation (13) defines the rest of 
the A^ consecutively. 

Notice that {T^}, like {S^} and {o^}, are homogeneous symmetric 
functions. 

Having chosen = 0, equations (10) can now be solved: 


°2i-l ^i-Zk-l* 7 ; 

k 


2k 


or 



(14) 


If, following Peterson, we now wish to find a solution of (14) with 
degree at most e, we set cr e+ ^ = tr e+ 2 = • • • = 0* Equations (14) can then 
be partitioned into 


if 
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The problem is thereby reduced to the solution of equation (15). 

This is equivalent to Peterson's equations in theorem 9.4, page 176. But 

our matrix of equation (15) is of dimension only (2[e/2] ) X (2[e/2]+ 1), 

and, furthermore, it is parasymmetric, having a constant value along 

any minor diagonal. This enables us to make certain statements about 

the existence and number of solutions to (15), because the enumeration 

of persymmetric matrices by rank has been accomplished by Daykin (I960) 

and Berlekamp (1963). The major result is that the number of mX n 

persymmetric matrices of rank less than or equal to r over GF(q) 
r 

is q , for any r < m < n. This enables us to conclude, for example, that 
equations (15) will have a unique solution for (q - l)/q = N/(N + 1) of the 
possible cases. This happens only when the square submatrix formed by 
omitting the first column of equations (15) is nonsingular. 

In fields with characteristic p > 2, it is very difficult to get 
similar results, for several reasons. One may introduce triangular 
transformations which reduce the dimensions of the matrices resulting 
from Newton's identities by a factor of p, but the resulting smaller 

s 
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matrices are not para symmetric and the problem of enumerating them 
has not been solved. The relevance of these cases to coding is also 
less direct, because one now needs both error-location numbers and an 
error number for each location to specify an error pattern. 
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MULTIPARAMETER SENSITIVITY IN LINEAR NETWORKS 
R. N. Biswas and E. S. Kuh 


Introduction 

In the analysis and synthesis of linear networks, a knowledge 

of the sensitivity of the network function under consideration with 

respect to the physical parameters is often extremely important. The 

1 2 

first definition of sensitivity was given by Bode ; Mason defined 
sensitivity by the inverse of Bode's definition, and that has been the 
accepted definition of sensitivity since then. The sensitivity of a 
scalar transfer function w(s) with respect to a parameter x is defined 
to be 


c w . - ^ n w _ 3w X 
x - 3in x c5x w 

However, this measures only the first order approximation of the 
change in w(s) due to variation in x, as is readily seen from the 
Taylor series expansion of w(s) around the nominal value of x. 

The purpose of this note is to give a compact formulation of the 
first order sensitivity functions, and use this formulation to obtain 
expressions for the "higher order sensitivity functions," which may be 
used to estimate the deviation of a transfer function from the nominal 
value with greater accuracy. 

Flow Graph Representation 

A convenient technique for investigating the sensitivities of a 
network function with respect to n varying parameters x^, x 2> . . ., x r 
is to represent the network by the following vector flow graph. 

The research herein was supported by the National Science Foundation 
under Grant GP-2684. 
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c 



I_ = Unit matrix of order n 
X= Diag. (x^ x 2> . . x n ) 

C= Transfer matrix from the outputs of x^, x^, ...» x^ to 

their inputs 


(c.^ = transfer function from the output of the j-th varying element to 
the input of the i-th varying element. ) 

The output vector ^ = Xv, where v = u + C y, u being the unit 
vector. The overall transfer matrix W, defined by the relation 
y = W u , is then given by 

w= (I - X C) _1 X . 


If u. and 
J 


y^ are respectively the scalar input and the scalar 


output of interest, the scalar transfer function is obviously given by 
w_ (i-th row-, j-th column- element of the matrix W). 


I 




* 
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The C matrix contains the fixed parameters of the network; 
if sensitivities with respect to all the parameters are sought, then C 
depends upon the network topology only. 

Let us briefly illustrate how a network can be put in the desired 
flow graph form. 

Example 1 ; Multiloop Feedback Amplifier 

Suppose we are interested in finding the variation in the overall 

transfer function due to changes in the individual stage gains. The 

stage gains are therefore labelled as x 1# x 0 , . . . , x . c.. is then 

12 n ij 

the net feedback factor from the output of the j-th stage to the input 
of the i-th stage. The form of the C matrix depends upon the 
configuration of the amplifier. 

Example 2; Passive RLC Circuit 

In this case all the parameters are considered to be subject to 
variations. If we pick a tree such that the scalar input is either a 
current source in parallel with a tree-branch or a voltage source in 
series with a link, and the scalar output is either a tree-branch voltage 
or a link current, and let 

v= Col. (tree-branch currents, link voltages) 

_y = Col. (tree-branch voltages, link currents) 

so that 

X= Diag. (tree-branch impedances, link admittances), 

3 

then it follows from Bashkow's results that 



where the fundamental cut- set matrix = (I • D). C is thus a description 
of the network topology only. 


The Sensitivity Functions 


The multiparameter Taylor series expansion of the transfer 

function w. . around the nominal values of x,, x~, . . x gives 

ij 1 2 n 


n 9w. . 


. n n 9 w. . 
1 V V* ij 


6w ij = X 9^ 6x k + h. X 


6x, 6x„ + ... 


k— 1 ^ k 


k=l 1=1 k" A f 


or 


6w. . n 9w. . x, 6x. , n n 9 w. . x. x* 6x, 6x„ 

ll = y IJ k k + _y y y IJ k 1 k l 

w. . 1 ^- 1 9x 1 w.. x, 21,^—, 9x. 9x „ w. . x, x„ 

ij k-1 k ij k k=li = l k ! ij k l 


or 


9w. . w., w. . 

U = lk kj ^ 

9x. 2 

k x. 

k 


=> 


w., w, . 

S X J _ ik kj 
k x, w. . , 

k ij I 


The first order sensitivity function is thus inversely propor- 
tional to the transfer function w^., and directly proportional to w.^ 

and w, which clearly have the following significances: 

KJ 

W ik = ex ^ raneous signal gain from the input of the k-th 
varying element to the scalar output 

w^j = signal gain from the scalar input to the output of the 
k-th varying element. 


Now, since W= (I - XC)' 1 ^ = (X _1 - C)'\ it follows that 




w. . = A.. / A , 
IJ Ji 


where A = det(X 1 - C) and 
the j-th row and the i-th 


A.. = the cofactor of 
J 1 

column. Then 


A 


corresponding to 


0 


S ij _ ^ki A jk 

k x, AA.. 
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This expression, which can be used for direct computation, 
clearly shows that both the poles and the zeros of the scalar transfer 
function are poles of the sensitivity functions. Thus in a synthesis 
problem, once the transfer function has been specified, the poles of 
the (first order) sensitivity functions are fixed; one has some control, 
however, over the zeros, and this fact may be utilized to minimize 
sensitivity while synthesizing a specified transfer function, by proper 
choice of the network configuration. 

The higher order sensitivity functions are obtained directly 
from definition, by repeated use of the relation 


9w. . w., w, . 

—1L = lk k J 

9x n w. . 

k ij 


for all i, j, and k. 


The results are the following. 


5 ij - c ‘J , ^ ij v £ ij W ik W kf W ij , .. 
S ki - + S ik ' where S ki = IT -x w (k 4 *>’ 

k i ij 


v-WGH— 


The coefficient of the first order terms are the usual (first order) 
sensitivity functions as defined earlier. We shall use the notation: 


c ij 4 
S k ’ 


9w. . x, 

3x, w. . 
k IJ 


The logical choices for the higher order sensitivity functions would 
then be the following. 
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2 nd order: 


3 w. . x n x fl 

s 1J ^ ij k j 

ki 3x, Sx. w. . 

k £ ij 

a 2 2 

, d w. . x, 

<r X J4 i 1L J*_ 

kk 21 rx 2 W. 


3x, 


1 J 


k 4 i, 


3 w. . x x .x 

3rd order: S 1 „ ^ =-5 k 1 ^— 5 : k ^ i 4 m 4 k and so on 

kim 3x, ox. 3x w. . 

k f m ij 

The factors -^y, -^y etc. are left out when the subscripts are different 

because the same combinations of different subscripts appear 21 
times in the 2nd order terms, 31 times in the 3rd order terms and so 
on; physically, all permutations of the subscripts for the same combina- 
tion have to be associated with one and the same sensitivity function. 

In terms of the sensitivity functions thus defined, then, the 
fractional change in the transfer function may be expressed as follows. 


Sw . . 
ij 

^T7 "^k 

ij k 


Is, 


. . 8x 1 

ij k 


I's 


. . 6 x. 6 x. 
ij k i 


^ k, i 


kl 


x k x i 


. . 8x, 6x. 6x 

s 1J k i m 

. . kim X. x.x 

k, £,m k i m 




+ . . 


The primed summation signs indicate that the summations are to be 
extended over all combinations of the subscripts, no distinction being 
made between terms involving the same subscripts in different orders. 

Expressions for the sensitivity functions are obtained through 
the use of the well-known matrix sensitivity relation: 

6 W = S 6 X X ' 1 W, 


relating the change 6 W in the overall transfer matrix to the change 
5X in the forward transfer matrix. For small 6 X, the (first order) 
sensitivity matrix is given by 
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S = (I - XC )' 1 = wx ' 1 
6W_= W x " 1 6X X" 1 W 

6w. . =y w.., 6x- — w, . . 

1J k lkx k kx k kj 

The expressions, though not very simple, clearly show a definite 
pattern; the higher order sensitivity functions are multilinear combina- 
tions of the first order ones, as shown in the following. 

s k/ j = s k ii s, ki + s t i \' i < k 4 

w - 4 *J\r ^ « 

and so on. 

These expressions clearly show that low values of for 

all k do not necessarily guarantee low S^ 1 ^, etc. for all k, 

i, . . . , except in the ideal case where = 0 for all k, in which 

case all the sensitivity functions are identically zero, so that the 
transfer function w^ is completely independent of all the network 
parameters. In general, therefore, unless the fractional changes 
6x^/x^_ are all small enough to justify a first order approximation, 
one has to take the higher order sensitivities into consideration. The 
advantage of the foregoing formulation is that this can be done without 
much complication, since the sensitivity functions are all expressed in 
terms of scalar transfer functions between different points in the 
network, which can be found directly from a given network topology 
and the nominal values of the parameters. 


- 15 - 



References 


a* 


!• H. W. Bode, Network Analysis and Feedback Amplifier Design , 
D. Van Nostrand Co. Inc. 

2. S. J. Mason, "Feedback theory - some properties of signal flow 
graphs," Proc. I. R. E. , Vol. 41; Sept. 19 53. 

3. T. R. Bashkow, "The A-matrix, a new network description," 
I.R.E. Transactions on Circuit Theory; Sept. 19 57. 


- 16 - 



' ON THE DEFINITION AND ANALYSIS OF 
PULSE-FREQUENCY MODULATED SYSTEMS* 

1 

Jean Gabriel Blanchard 

I* DEFINITION OF A GENERAL PULSE -FREQUENCY MODULATOR 
(GPFM) 

1. Introduction 

A GPFM is defined as a system operating on piecewise continuous 
inputs and converting them into sequences of asynchronous pulses of 
identical shape but different signs. The output of the modulator is 
completely characterized by the sequences {t }, {« }, and the func- 
tion PqU) (Fig. 1). The input-output relationship of the GPFM, 
m(t) = M [x(t)] , where M is a nonlinear operator, is in fact a relation 
between x(t) and the pair ( { t^} , {e^}). 



Fig. 1 



The research herein was supported by the University of California 
Fellowship Program. 




2. Definitions 


A. The input x(t) is assumed to be piecewise continuous. We define 
x(t, 0 1 / 0 ) as follows (Fig. 2): 

r 0 for t < 0 

x(t,0'/0) = < x(t) for 0<t<0' 0 f >0 

„ 0 for t > 0 1 



0 0 ! 0 0 ' 0 1 


Fig. 2 

B. We shall denote by P (t, t ) a function (or generalized function) 

U; j p 

where t^ is a variable and (a, \ ) are parameters. Assuming t to 
be known, we have: 




for t < t 


P i(f> t ) = < P n (t - t ) for t < t < t + a 
a, i p 0 p P P 

0 for t > a + t 


a - P Q (t) may be a given function of t equal to zero for t < 0 
and t > a. It is continuous in the interval (0, a) (Fig. 3). 




p o (t > = e 


• e 


(t-aV 


for 0 < t < a 


Fig. 3 


b. Pp(t) ma y also be a generalized function: P^t) = 6( t) . This 

case can be considered as a limiting case of a. (see above). Take 
P Q (t) = \[u(t) - u(t - a)] where a is small and \ large. Assume cp(t) 
to belong to space D. We have then: 


f +c° a 

/ <p(t)P (T)dT <p{ T)d t = \a<p(£) o < £ < a 

J - co u 1 0 


and P (t) is equivalent to H6(t) where \a - H. 

C. Consider now the class of all functions of the form x(t, 0 ' / 0 ) where 
9, 9* are arbitrary but finite and satisfy the following inequality: 

9 < 9 < 9'. We define then a real functional F whose domain is the 
previous class and which takes values on the real line. We denote the 
value of this functional by 9 ’) . The following assumptions are 

made on I x ^(0 f 9*): 


W e ' 9 ' ) = 0 


for x(±) - 0 in (0, 0 ') 


wv 0 ' 1 ' 


for fixed 9^, is a continuous function of 9' 


for all x( t) . 


I x(t )(e.0) = ° 
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/ +oo , 0 1 

Example: I .(0,0')= / K(t)x(t, 0 0)dT = / K(t)x(t)<1 

'-oo J 0 


D. 


f T, is a given sequence of positive numbers. 
1 k jK=0, 1, ... 


3. Definition of M 

Assume x(t) to be given. Consider an arbitrary t^. We define 

the output m (t) of the GPFM, corresponding to the input x(t, t n /0), 

r 0 U 

as follows: 


m 



(t, t ) 
n 


( 1 ) 


where PQ(t), a, \ are given and the sequences {t^} and € {t^} are 
determined by the following relations: 


A. 0 < t x < t 2 < 


< t < . . . < t n . 

n 0 


B. Assuming t . to be known, t is defined as the first value of t 
° n-1 n 


for which 


^x(t/t*n-l, 


t) = T 


n-1 * 


Furthermore, e(t ) is taken equal to 


+ 1 if I ,, x {t . t } > 0 or -1 if I . .{t , t }<0. 

x(t) n-1, n J x(t) n-1, n J 


C. Conditions A and B determine uniquely m (t) for a given x(t). 

t 0 

D. The output m(t) corresponding to the total input x(t) is defined 

as limit of m (t) when t n — > oo . 

t 0 U 

4. Example 

Consider the system represented in Fig. 4. Assume the GPFM 
to be defined as follows: 
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x(t) 

GPFM 

m(t) 

1 


4 



2 

s 



v(t) 


Fig. 4 


6 T 

1(0, 0') = / x(T)d 
1 0 


T k = 


x(t) = 


(k + 1)‘ 

ro 

-pt 


le 


for t < 0 
for t > 0 


P 0 (t) = 6(t) 


Determination of the sequences (e } and {t }. 

P P 


1 ( 0 ' 


■,e)=/ e > T 4 t.‘ 

* e H 


-P0 -pe 


- e 




0' > 0 -^-1(0', 0) > 0 so € (t k )=e k =+l for all k. 


The t k are determined recursively by the relation: e - e 


_(3t k-l " Pt k_Tp 


^ P l 

We have therefore: 1 - e P = y — >r or = ^log n 


— “T 

c=l k 


k=p ' 

* - $ 
k=l k^ 


Two cases may be distinguished: 

k=oo 
£ k 


A. T < ^2 or ^ ^ > 0: there is an infinite number of emitted 


pulses . 
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B. T > — 


or 



there is a finite number of emitted 
where N is the greatest integer such 



A GPFMS is a system having one or more GPFM as components . 
It contains linear and eventually nonlinear plants. Feedback loops may 
also exist (Fig. 5). 


I 



Fig. 5 


Large systems as defined previously are difficult to analyze. Only 
simulations on digital computers or approximation methods seem to be 

of interest in their study. Some simpler cases (Fig. 6) will be studied 
in details . 



Fig. 6 


f 
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III. CASE OF A GPFM FOLLOWED BY A LINEAR PLANT DESCRIBED 
BY A VECTOR DIFFERENTIAL EQUATION OF THE FORM 


Y = AY + Bm( t) 

1. Determination of the linear plant output [P^(t) = 6(t)] 


Assume the linear plant to admit an input-output relationship of 


¥ N (k) 

the form 2_ ay' ' ~ m (t) 


k=0 


N = 1 


N-l< 


Take A = 


0 

1 0 . . . 



0 

0 

0 1 . . . 


- B = 

0 

0 

0 

1 


• 

_- a 0 

V. . 

- a l 

" a N-l_ 
. . , > 


1 


~N 



- - 


Y(t) = e^*" " *0^ Y(t Q ) + e At J e AT Bm(r)dT 

t 0 


but 


At k =m , 

e = Y a (t)A 0 < m < N - 1 

k=0 k 


take 


t„ = t and t + < t < t ■ , 

Op p — - p+1 


m(t) = c(t p )6(t - t ) 


A (t - t ) At Hp- m k r l 

Y(t) = e P Y(t )+e Y A k B / e (t ) or, (t)6(t - t )dT . 

“ P k=0 ~ J t p k P 


So 


A(t - t ) 

Y(t) = e P [Y(t" ) + e(t)B] 

r -H 


«(t )<*. (t ) 

p k p 


( 2 ) 
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Now set t = t Y = Y(t ). We obtain from (2): 
p-rl — p — p 


A(t t ) 

Y . = e P P (Y + e B). 

— P+ 1 — ’P p — 


( 3 ) 


This discrete system is completely defined if we know the sequences 

(t } and {e }. Conditions 3. A. enable us (at least in theory) to 
P P 

determine t (1 and e as a function of x(t). Furthermore, Eq. (3) 
P+1 P 

suffices to characterize the output of the linear plant (Eq. 2). 


2. Determination of {t } and {e } 

e : p_ 

The main difficulty lies in the solution of the equation Ift^, t) = * 


A. Example: 


Take x(t) = a + bt + ct . Assume 




t 

k( t)x( T)d t . 


n 


From 3. A. we deduce: 



n 

or if we set 


t = 


€ n+1 


T 

n 


f t f t 

I k(r)dT = A(t, t n ), / tk(T)dT = B(t, t n ), 

^n ^n 


/ 1 2 

I t k(r)dT = C(t, t n ) 
^n 


aA(t, t ) + bB(t, t ) + cC(t, t ) = € - T. 

n n ; n' n+1 


Finally we obtain a relation of the form f(t, t ) = € t1 T where t , 

n n+1 n n+i 

is the smallest positive root of the previous equality. In general, it is 
impossible to find a closed form expression for t j as a function of 


t^ ( similarity for the computation of € ). 


Tn. 
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B. If there is a feedback loop, the problem of finding the t and £ 

n n 

becomes also unsolvable in general if we do not use approximation 
methods, graphical solutions, analog or digital simulations. For instance. 


I(t . t) =/ f[e(T)]dT. 
't 

n 


We then have: 


k-m 

Y. = Y O', (t - t )A k [Y + € B] 
““1 £tq k x n L — n n J 


for t + < t < t” r - 
n n+1 


Take 


Y + € B = z 
— n n — n 


a, = first row of A 
— k 


k=m 

= X - l n > < 2k' in 


k=0 


owing to the form of A, reduces to 


0 

0 


(k + l)-th row 


k=m 


and = j a, (t - t z, . The t and € are obtained from the 
7 1 k v n' k, n n n 

k=0 


relation 


k=m 


f i i jx— 

I t ” £[U(T) ‘ jlj “k (T ' *n> Z k, n ldT= '„+l T n- 

n 


(4) 


- 25 - 



This equation in is not in general solvable: 

in theory 


t f1 = f(z , t ) 
n+1 — n' n' 


€ , = g(z , t ). 

n+1 — n n 

An interesting case arises when I(t^, t) is taken equal to 


I x( t)c!t 


n 


and the sequence T^ reduces to a constant threshold T. This particular 
case has been previously studied under the name of IPFM (see references). 
Few general results appear to have been found. From Eq. (4), it may 
be seen that for zero input [u(t) = 0], the relation determining the t 

k=m . t j 

and € is of the form -3" / n or. (t - t )z, dT = € , T or if we set 

P J t k n k > n n+1 

t ’ n 

7 n+ V T - *„> = A k<*n + 1 - l „» - A k (0 » and ‘n+1 - ‘„= fit n 


' t 


k=m 


X fA k ,at „» • A k ,0| K, n= € n+1 T ' 

k= 1 


Therefore, we may write: 


At = f(z ) e i = g(z ) . 

n — n n+1 ° — n 


(5) 


The discrete system considered in part III. 1. becomes then an autonomous 
homogeneous system: 

Af(zJ 


Y . = e 
-P+1 


-P 


(Y + e B) 
-P P“ 


or 


-P+1 


Af(z ) 

e (_z p ) + g(z p )B . 


The stability of this system may then be studied using an extension of the 
Liapunov theorems if a closed form expression or a valid approximation 
may be found for Eq. (5). 


- 26 - 



IV. ZERO INPUT ANALYSIS AND STABILITY IN THE CASE OF A 
FIRST ORDER LINEAR PLANT WHEN THE GPFM REDUCES 
TO AN IPFM 

Consider the system represented in Fig. 7. 



Fig. 7 


For t + < t ^ t" . 
P ~ P+1 


y(t) = e 


-a(t - t ) 


(y + « ) 
P p 


-a(t . - t ) 

y . , = e 15 ^(y+e) 

p+1 ' r p p' 


1. Determination of the {t } and {e } 
IP" P 


( 6 ) 


-4 


t . -a( t - t ) . f 

P+1 e P (y +e )dT= t ,.TS--le 

P P P+1 a 1 


-al-r-lttp+i 8p+1 T 

J*P ' V*P 


or 


; " a(t p+l " V = n aT c 


y + e p+1 • 
P P 
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“ 3 . t 

Since e (for t > 0) is <1 we have 


€ - sign (y + € ) 

p+1 w p p 


( 7 ) 


and if y + e > aT 
1 P P 


-a(t , - t ) ~ 

s = i - si g n (y + O 


p p y + « 
p p 


Finally: 


t . , - t = - — log. T f 1 - sign (y + e ) ^ ^ 

p+i p a s nV s y P p'y p + « p / 


( 8 ) 


2. Determination of the "equivalent" discrete system 
From Eqs. (6), (7), and (8), we deduce 


y p+l Y p + ‘p 


(sign (y p + « p )j a-T. 


Now sety + e = z (z = y(t )). We obtain 
P P P P P 


z , - = z - (aT + 1) sign z . 

p+i p s p 


(9) 


with 


p+1 


-sign z 


t 1 - t = -- log (l - -+n-') 
P+ 1 P a V 1 2 J / 


r 3 * * 6 o = 0 
t 0 = o 

v z o = y o 


3. Stability 

Two cases may be distinguished. 

A. Zq > aT + 1 or < - (aT + 1). For instance take z^ > 0. Owing to 
Eq. (9), sign (z^) = + 1 = -1) during p steps where p is the largest 

integer such that 


rrTT ' 1 - t -1YTT 
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at the p-th step, z^ = Ej is in the interval [0, aT + 1] 


a) Ej < aT. 

No other pulses will be emitted. The system stops at the p-th instant. 

b) aT < E l < aT + 1 and aT > 1. 

Z p+1 is < 0 and is in the interval [-aT, 0] let E 2 = z r No other 
pulses are emitted; the system stops at the (p+l)-th instant. 

c) aT < < aT + 1 and aT < 1. We see that 

Z p+Z = Z p+1 + + ^ = Z p = Ej. will take therefore the values 

and E 2 alternatively and the total output will oscillate between E^ and 

E 2 / -a(«-t ) _ \ 

Ut = ' % f or t p+ < « <t p+J . 


The period of the oscillations is 


P = 


‘ - l°g* T 
a 6 N 


1 - 


aT 


1 - 


aT 


(Fig. 8). 


m(t) 


modulator output 

steady state 



transient 

state 


Fig. 



B. I z Q 1 < aT + 1 


Similar conclusion without transient part. 
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THE PROBLEM OF NEURON MODELING 
J. G. Blanchard and E. I. Jury 



I. THE PROBLEM OF SIMULATION OF NEURONAL BEHAVIOR 

Several attempts have been made in the past to simulate by 
electrical networks the behavior of neural elements. Physiologists^ 
have tried to match their experimental results with partly empirical 
formulas describing the variations of the main chemical or electrical 
factors taken into consideration. Engineers in the field of bioengineering 
have directed their efforts toward developing models reproducing 
qualitatively the main electrical properties of the neuron. 2 ’ 3 

A brief survey of the literature shows that connections between 
the work of those physiologists and bioengineers are not always as strong 
as it would be desirable. For some reasons due mainly to the complexity 
of the phenomena to be studied, the different models presented by engi- 
neers appear to lack a certain unity which would have been preserved if 
more attention could have been devoted to the assumptions, approximations, 
and limitations involved. Furthermore, as the models become more 
and more sophisticated, the risk of making speculative assumptions 
increases unless the physiological aspects of the problem are constantly 
kept in mind. It appears finally that the models in question do not display 
enough degrees of freedom to take into account most of the parameters 
influencing the neuronal behavior. 

It is our opinion that the problem of neuron modeling ought to be 
rethought if the work of bioengineers is to take a larger place in this 
particular domain of neurophysiology. Further research could be 
oriented in two different directions: 

The research herein was supported by the Air Force Office of Scientific 
Research under Grant AF-AFOSR-292-64. 
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A. Development of accurate models based on quantitative and 
qualitative experimental data about the chemical and electrical behavior 
of the precise neural element to be studied. Owing to the nonlinear and 
time-varying character of the phenomena involved, it is likely that any 
model of this sort will necessarily be complex and defy any deep analysis 
with mathematical tools now available. However, a computer study is 
always possible. Any such attempt should require the combined efforts 

of physiologists and engineers. It would be particular ily helpful if further 
investigation were made of the neuronal properties as a function of dif- 
ferent parameters, such as temperature or external chemical concentra- 
tion. 

B. Development of simplified models, in which more initiative 
could be left to the bioengineer, in order to investigate very complex 
problems such as those arising in the behavior of groups of neurons or 
synapses. Any approach to those problems should require definitions 
and methods not always available to the physiologist. Optimal models 
could then be defined with respect to certain performance criteria 
corresponding to the particular behavioral function (or functions) to be 
studied. 

II. A MODEL BASED ON THE HODGKIN AND HUXLEY EQUATIONS 

In 1952, A. L. Hodgkin and A. F. Huxley proposed an electrical 
circuit (Fig. 1) to represent the membrane of an axon. ^ It is our 
opinion that this model is as important today. 
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0 

0 

0 


A. Definitions and Notations 

I and E are, respectively, the transmembrane current and 
potential. E^ a and denote the sodium and potassium equilibrium 

potential as calculated from the Nernst equation. E^ is known as the 
leakage potential and takes into account other ions such as clorine ions. 
E r represents the resting transmembrane potential. §N a ’'®K’ anc * 

g^ are the sodium, potassium, and leakage conductances. C.^ is the 

transmembrane capacitance. We now set: 



V 


Na 


< 0 , 


V K > 0, Vj < 0 
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B. Analysis of the Circuit 

I and V are variables. and g T ^ are function of V and t. 

& Na fe K 

The other quantities are parameters. 

Basic equation: 

1 = C MHT + 6 N a< V ' t)(V " V Na> + S K < V ’ '>< V ' V + «1< V ’ V l>' 

The quantities g^ a an d gpr are determined as follows: 
g Na = g i m h 

g K = § 2 n4 

where and are constants and m, h, n, are variables determined 

by a set of time-varying first-order differential equations of the form: 

■ 3 — = a (V(t))*(I - x) - b (V(t))-x for x = m, n, h, (2) 

at x x 

where a^ and b are known functions of V(t) determined empirically 

(see appendix). The constants x(0) and (x = m, n, h) are also determined 
empirically in order to fit experimental results. 

The main difficulty arising in the analysis of these equations is 
due to the variation of g ^ and g^. Theoretically, it is possible to 

solve equations(2) and express their solution as a function of V(t), t and 
the initial values x(0). If we now insert these solutions into equation (1), 
we obtain the following results: 

i) V(t) = H(V(t), t) + kl(t). 

This form is of interest if I(t) is considered as being the input and V(t) 
is the output of the system. 
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ii) I = G( V(t), t) + C v 

M 

This relation would give the output I(t) as a function of the input V(t). 

For example, this equation would enable us to determine the trans- 
membrane current when V(t) is constant (Simulation of Voltage Clamp 
experiment) . 

C. Model Properties 

It has been shown by Huxley and Hodgkin that equations (1) and 
(2) describe satisfactorily the membrane behavior when a stimulus 
(current stimulus or initial value 4 0 imposed to the transmembrane 
potential) is applied. The phenomena of potential threshold, current 
stimulus threshold, refractoriness (absolute and relative), accommodation 
anode break excitation, and so on, have been correctly reproduced by 
these equations. 

D. The Problem of Local Excitation by Current Stimuli and Its Analog 

Computer Simulation ~~ 

This problem is of particular interest in the study of the receptor 
(or generator) potential and repetitive firing relationship. It is generally 
assumed that the current induced by the generator potential plays the role 
of stimulus in generating the repetitive discharge. 

An analog computer model based on equations (1) and (2) enables 
us to simulate the I(t), V(t) relationship (see below). The interest of 
such a simulation is obvious. The main chemical and electrical factors, 
such as or g^ or even some empirical relations such as a m (V), 

can be easily modified and adjusted. This type of model could be con- 
sidered as belonging to the class A models described in Part I of this 
paper. 
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III. A SIMPLIFIED MODEL USING THE PROPERTIES OF PULSE 
FREQUENCY MODULATED SYSTEMS 4, 5 ’ 6 

We present now a simplified model which could be of interest 

in the study of sensory receptors and the transmission of information 

to axons. It is generally accepted that the original stimulus causes a 

deformation of the dendritic branches of the sensory receptor. A long- 

lasting depolarization of the resting potential of the receptor known as 

generator potential follows. The generator potential acts as a persistent 

cathodal stimulus generating currents which cause the initial segment 

7 

of the axon to respond repetitively to the depolarization. (See below) 

A. Analysis of the Model 


i) Definition of the transducer: The transducer is assumed to 

be a linear system converting the original stimulus (e. g. , mechanical) 
into an electrical stimulus (generator potential). For large e(t), there 
is a saturation. 


V(t) = j h(t - u)f(e(u))du, 

1 0 

g 

where f(e(u)) is represented in Fig. 4. In a first approximation, h(t) 

— a t 

can be taken as e with a > 0. 


f( e ( t) ) 



Fig. 4 
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ii) Definition of the PFMS (Pulse Frequency Modulated System). 


This element converts the information provided by the generator potential 
into a sequence of pulses (action potentials) called the repetitive discharge. 

a) Notations: 



- V 


• V(u)du 


T k (t) 



bk . 
e / 




P l, h (t " V ' h * P(t " t k ) 

where c, b, q, h, t^ and are positive constants, 

t, is the time at which the k-th pulse (action potential) is 
emitted, 

V(t) is the generator potential, 

P(t) is a continuous function of time t, equal to zero outside 
a certain interval (0, d) where d is positive (Fig. 5), and 

f(t k ) =0 if k = 0, 

= 1 if k > 1. 



Fig. 5 
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b) Let us clarify now the meaning of those p arameters from the 
point of view of physiology: ~~ ~ 

c refers to the delay separating the beginning of a stimulus above 
threshold and the time of emission of an action potential. Further- 
more, it will determine the value of the rheobase if the strength- 
duration curve is studied. 


Tq can be described as being the value of the membrane threshold, 
when no recent stimulus has been applied. 


b represents a quantity which can be taken as the rate of adapta- 
tion of the receptor (b is in general very small). 

q and the curve 1 / ( 1 - e t r^) represent the variation of 

excitability (inverse of the threshold) after the period of absolute 

refractoriness following the emission of an action potential. They 

refer therefore to the relative refractoriness. The coefficient 

f(t^) = 0 or 1 enables us to study separately the case where no 

recent stimulus has been applied (the membrane potential is equal 

to its resting value). The quantity t-t, -t can be considered 

.K. r 

as being infinite and the threshold T^(t) reduces to Tq . 
t represents the absolute refractory period. 

P(t) describes the form of the action potential and the coefficient 

h enables to adjust its magnitude. The origin of the potential 

axis is taken at -E f where is the resting value of the membrane 

potential. 

V(t) is always assumed to be positive. We only consider the case 
of depolarizing inputs. 


c) Analysis of the input-output relationship of the PFMS 

Assume an action potential to have been emitted at time t^. The 
value of the time t^j at which the next action potential is to be emitted 
is the smallest positive root of the equation: 


I(t, 


w = 


T k<‘>- 
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In general, we can write the output as: m(t) = V P. ,(t-t ) where 

+ 1, h k: 

k 

m(t) represents the repetitive discharge of action potentials. 

Example: Set V(u) = V = constant. Consider k = 0 (first action poten- 

tial) . We then have V(1 - e _t ) = c • T Q . 

For the following action potentials, we have: 

-(t-V -q(t-t,-t ) 

V( 1 - e K ) = c • T q /( 1 - e k r )(-), t k <t<t k+1 . 

d) We shall now prove that this model agrees with the main 
properties of the neuron. 

Strength duration curve 

For V < c- Tq, there is no excitation. The rheobase is therefore 
equal to c* Tq. The equation of the strength duration curve is 

V = c* T /(I - e ) (see reference 9). 
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Gradie nt-threshold curve 

Assume V(t) = st where s is a constant. We have 


I (t, 0) = ( -s / c)te” Ct + (s/c 2 )(l - e" Ct ). 


I ( t, 0) is monotone increasing and its maximum is s/c . We see 

2 2 

therefore that for s<c there cannot be excitation. This value c T 

corresponds to the gradient threshold. See Fig. 7 below. 


gradient-threshold curve 


Fig. 7 


Absolute refractoriness 


Assume that an action potential has been emitted at time t^ ar 
that V(t) is a constant equal to V. 

As we have seen, the following equation enables us to compute 
the time t^^ em i ss i° n °f the next action potential: 

-(t-t ) -q(t-t -t ) 

V(1 - e K ) = c- T q /(1 - e ). 


We see that, whereas the left inside of this equation is always positive 
for t > t^> the right inside will only be positive if t > t^+ t^. In other 

words, there cannot be any root of the equation I(t, t, ) = T^(t) lying 

in the interval (t, , t, + t ) . t is the absolute refractory period. 




Relative refractoriness 


Figure 8 shows the variations of T, (t) for t > t. + t and the 

k k r 

variation of I(t, t^) for t > t^. The variation of the threshold is in good 
agreement with experimental data (see reference 9). 



Fig. 8 


Adaptation 


We see from (#) that when k 

increases and t. , . - t, increases. 
k+1 k 


bk 

increases (for V constant), e 
This property corresponds to the 


well-known property of adaptation (Fig. 9). 



Fig. 9 


A -* t +t, 
r k 


B 

C 


* t k 1 + 1 

'V 1 
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B. Interest and Limitations of the Model 

The digital simulation of the model is extremely easy. Further- 
more, the parameters of this model can be adjusted to agree with 
experimental data (for example, F = 2 ms, d = 6 ms, h = 95 mv) . Finally 
this model reproduces the main properties of neuron excitation. It 
could be noted, however, that this model does not intend to provide the 
values of a subthreshold potential. However, I(t, t k ) is a good approxima- 
tion to this potential if accommodation for subthreshold inputs is neglected. 
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APPENDIX: 


a n = 0. 01 (v + 10)/(e"T0~- 1) 


b = 0. 125 e 
n 


v+25 


a m = (0 . 1) • (v + 25)/(e - 1) 


b = 4 e 
m 


~ v/20 

a, = 0.07 e 
h 

v+30 

b, = l/(e 10 +1) 


v in m volts. 
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ON THE STABILITY OF FEEDBACK CONTROL SYSTEMS 
WITH PERTURBATION GAIN* 

C. T. Chen 


Abstract 

This paper considers a special class of single-loop feedback 
systems (such as shown in Fig. 1) both in the continuous and in the 
sampled-data cases. It is shown that if the gain deviation from the 
linearity of the memoryless nonlinear element, $>(cr, t), is smaller 
than \(t)tr, then \( • ) € L (0, oo) implies that the system output is 
bounded for any bounded input. As t— *oo, the system output tends to 
that of the linearized ($>(cr, t) = 0) system as t— >oo. Similar results 
hold for the sampled-data case. 

I. Continuous Case 

Given the system 



Fig. 1 

where N is a memoryless, time-varying, nonlinear element, G is a 
linear, time -invariant subsystem. 

Assumption 

(A. 1) The input-output relation of G is given by 


This work was supported wholly by the Joint Services Electronics 
Program (U.S. Army, U.S. Navy and U. S. Air Force) under Grant 
No. AF-AFOSR-139-64. 
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( 1 ) 


y(t) = 



- t)x(t)<1t 


where y(t) is the output, x(t) the input, z(t) the zero-input 
response, and g(t) the unit impulse response of G. 

(A. 2) For all initial states, z(t) is bounded on [0, oo) 

(A. 3) g(t) = l(t)[r + g-^(t)] where l(t) is the unit step function; r > 0; 

g^(t) is an element of 1^(0, oo), bounded and 0 as t — oo. 

(A. 4) The nonlinear, time-varying element N is characterized by 
x = (p{<r, t), and <p( cr , t) = kor + <p(cr, t), k > 0 where ^(c r, t) is 
the nonlinear effect due to gain deviation. 

Theorem 1. Suppose the feedback control system shown in Fig. 1 
satisfies ( A. 1) ~ (A. 4), and if 


(a) 1 + kG(s) 4 0 for all Re s 0 

(b) there exists a continuous, nonnegative, real valued function 
k(t) such that 


| <p((T, t) | < \(t) | CT | , 


and 



oo 

\(t)dt 


< N < oo 


then 

(i) for all bounded inputs u, the output y is bounded 

(ii) for any output y(t), there exists a response ^(t) of a 
linearized system [by putting <p( cr, t) = 0] which has the 
same input and initial state, such that 


lim | y(t) - 'y(t)! = 0. 

t ““ OO 


Pr_oof: If <p( cr, t) = 0, and (A. 1) - (A. 3) and (a) are satisfied, the 

closed loop unit impulse response, h(t), of the linearized system will 
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be an element of L^O, oo), bounded, — 0 as t -* oo.^ It follows, 
that for any bounded input, the output y(t) will be bounded. 

Introducing h(t) and y(t), Eq. (1) can be written as 


y(t) 


i r t 

= y(t) + j -J h(t - T)?[u(T) - y( t), T]dr 


( 2 ) 


1 


OO 


where | h(t)e~ St dt = y ^ s ) . 


Let ' sup h M - sup|h(t)|, = sup|u(t)j, then 


0<t<oo 


M 


M 


y(t) i £y M + k 


1 1 1 

- |h(t- t)|\(t)|u(t) - y(j)|dT 

v > n 


A , 1 


hif' 

k Jo 


“ y M + k h M U M N + MT)|y(r)|dT. 


h. .N 
M 


1 " LVJ 

Apply the Gronwall- Bellman inequality; we get | y(t) | < (^+ k h M U M N ^ e k 


i.e., y(t) is bounded on [0, oo). Define y^ = sup y(t) . From Eq. (2) 


i r t 

y(t) - ^(t) | - kJ 0 I - T ) I l^[ u ( T ) - y( T ). t] | d- 

-f | h(t - T)|\(T)dT. 


U M + 


Because h(t) is bounded on [0, oo), tends to zero as t oo, 

and \(-)*L^/n \ from Lemma 1, it can be concluded that 

(0, oo) 


y(t) - ^(t)| -"0 as t oo. 


Q.E.D. 


Lemma 1. Let £, be bounded on (0, oo), -*■ 0 as t — oo, and v ; 

. i z (0, oo) 

then fj* ~ 0 as t ^ oo, 
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Proof: For any € > 0, 3 t and T' such that | f^( t) | < € for all t > T 



for t > T 1 . Q. E.D. 

Corollary 1. 1. Let (A. 1)~(A.4), (a) and (b) hold. If, in addition 
r > 0, z(t) -*■ z^ as t — co, and if u(t) is bounded on [0, oo) and tends 

to u as t oo, then the output y(t) will tend to u as t -* oo. 
oo oo 

Proof: From Ref. (1) ‘ylt) — u^ as t — oo. Apply Theorem 1. 

y(t)-*uast — oo. Q.E.D. 

oo 

Corollary 1. 2 . Let (A. 1) ~ (A. 4), (a) and (b) hold. If, in addition 

r > 0, z(t) — ■ z as t — oo, and if then y(t) — 0 as 

’ ' ' oo (0, oo) 

t — oo. 

Proof: This is a direct application of Theorem 1, Lemma 1, and the 

fact z(t) — z^^ f h(t - T)z(T)dT — z^ as t -* oo. 1 Note: u( • )e L'^ ^ 

u(t) -■ 0, as t oo. 

II. Sampled-data System 



Fig. 2 
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It 


where N is a memoryless, time-varying nonlinear element, G is 
a linear, time-invariant subsystem. The sampling period, T, is 
assumed to be constant. 

Assumption 

(B.l) The input-output relation of G is characterized by 

n 

y(n) = z(n) + g(n - m) X(m) 
m= 0 

where the sequence {y(n)} is the sampled output, {x(n)} the 
sampled input, {z(n)} the sampled zero-input response, and 
{g( n )} the sampled unit impulse response. 

For all initial states, {z(n)} is a bounded sequence. 

oo oo 

g*(t) = X g( n ) 6 (t - nT) = T [r + gl (n)]S(t - nT) 
n=0 n=0 1 

oo 

where r is a nonnegative real-valued constant; |g (n)|<oo, 

n=0 1 

a condition which may be written (g^n)} 

The nonlinear element is characterized by x = cp (cr , n) and 
<p(cr, n) = kcr + <p(cr, n), k > 0 

where !p( cr, n) is the nonlinear effect due to gain deviation. 
Definition: 

G*( s) -«£[g*(t)] = f g(n)e' nTs 
n=0 


( B. 2) 
( B. 3) 

( B. 4) 


H*(s) = 


kG*(s) 
r+ kG*(s) 


/I 1 ^ 

h*(t) = *£ [H*(s)] = y h(n)6(t - nT) 

n= 0 
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Theorem 2. Suppose the sampled-data system shown in Fig. 2 satisfies 
(B. 1) ~ (B. 4), and if 

(a) 1 + kG*(s) 4 0 for all Re s > 0 

(b) there exists a nonnegative real-valued sequence {X(n)} 

CD 

such that | £(<r, n) | < \(n) | cr j and ^ X(n) < M < oo; furthermore, 

n=0 

h,. 

0 < -£— -\( n) < 1 for n = 0, 1, 2, . . . , where 

JK 

h M~ maX 
n 

then 

(i) for any bounded input sequence (u(n)}, the output sequence 
{y(n)} is bounded. 

(ii) for any output {y(n)}, there exists a response y(n) of a 
linearized system [ by putting <p(r, n) = 0] which has the 
same input {u(n)} and initial state, such that 

lim | y(n) - y(n) | = 0. 
n — oo 

In proving this theorem, we need the following lemma. 

Lemma 2. Let {g(n)} , {\(n)} be real-valued sequences, c, d be 
real constants. If {X.(n)}€ ft and 0 < d\(n) < 1 for n = 0, 1, 2, . . . , 
and if 

g(n) < c + d Y Mm)g(m) 
m=0 

then there exists a positive constant P > 0, such that 

a/ < - . — ... — — ^ ^ oo for n — 0, 1, 2, • • . 

&v ' — n p 

TT t 1 " dX.(m)] 

m= 0 

Proof: This lemma will be proved in two steps. 
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! 


n 


(i) g( n ) < c + d ^ \(m)g(m) < — — 

m=0 


for n = 0, 1, 2, 


TT t 1 ” d\(m)] 

m= 0 


If n - 0, it is obvious. Suppose when n = k, it is true, 
then when 

k 

n = k -f 1, [l - d\(k + l)]g(k + 1) < c + d k(m)g(m) 

m= 0 


; (i) is proved by induction. 

TT [1 - d\(m)] 

m=0 


(ii) 


oo 

0 < (3 < TT [1 - d\(m)]. 


m=0 


1 00 1 
Because {\(n)}el, there exists an N, such that d \(m) < -i 

m= N 2 

oo oo 

It implies TT [i - d\(m)] 1 - d V X(m) > y. 

m= N m= N 


N-l 

Let TT [i - d\(m)] = 2(3 > 0 

m=0 

then 

co 

| | [l - d\(m)] > (3 > 0. 
m= 0 

n 

Because | | [l - dX(m)] is monotonically nonincreasing and it is 
m= 0 

bounded from below, the limit exists. By (i), (ii) the desired result 
follows. Q.E.D. 

Proof of Theorem 2. Before we start to prove the theorem, some well- 

known facts will be recalled. Consider the linearized system shown in 
Fig. 3. 
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in the closed right half plane, it follows {h(n)}€j^. Let "y(n) be its 


1 

output, then 'y(n) = z(n) + T h< n - m)[u(m) - z(m)] . Since {h(n)}e<, 

m= 0 

for any bounded input sequence and bounded sampled zero-input response 
of G, the output {^(n)} is bounded. 

Introducing h(n) and 'y(n), Eq. (3) can be written as 


1 n 

y(n) = y(n) + s- ]T h(n - m)^[u(m) - y(m), m] 


Let 'y. - max y(n) , h . - max h(n)|, then 


y( n )|<y M + iJh M u M + -^ f Mm)|y(m) 


Direct application of Lemma 2: {y(n)} is bounded. Let 

y M ■ max l y( n ) I • 

n 

To prove (ii), we need the following well-known fact; if 

{\(n) } e j? 1 , {h(n)} e jf 1 then {\(n)* h(n)} € f 1 and \(n)*h(n)-*0 as n->oo. 
Since 

| y( n) - $ n) | < M ^ — — ]>[ | h( n - m) | \( m) . 

m= 0 


It follows y(n) - y(n) — > 0 as n — > oo . 


Q. E.D 




Corollary 2.1. Let (B.1)~(B.4), (a) and (b) hold. If, in addition 
r > 0, z(n)— >z as n— >oo, and if {u(n)} is bounded and u(n)— >u 

00 OO 

as n— > 00 , then y(n) tends to u as n— >oo. 

CD 

Proof: For any c > 0, there exists N, such that \z(n) - z l< c. 

1 oo 1 ' 

|u(n) - e for n>N. Since {hfnjjd 1 , for the same e, there 


N 

exists N 1 > N, such that V |h(n - m)|< e, for n > N 1 > N. If r > 0, 

m=0 


00 

Y h(n) = G* (0) = 1, therefore, for 
n= 0 

n 

1 - « < ^ h(n - m) < 1 for n > N". 

m=N 


€ > 0, there exists N", such that 
oo 

Let P = V |h(n)|, then 
n=0 


. N 

y(n) = z(n) + Y h(n 
m= 0 


n 

m)u(m) + J h(n 
m=N 


m)u(m) 


N 

y h(n - m)z(m) 
m= 0 


n 

- Y h(n - m)z(m) 
m= N 


N n n 

- Z oo +6 + U M Z |h(n-m)| + Uoo ]T h(n - m) + e Y |h(n-m)| 
m=0 m=N m=N 


N 

+ Z M Z l h ( n " m > 

m- 0 


z 

oo 


n 

h(n - m) 

m= N 


n 

+ € Y | h(n - m) | for n > max (N\ N") 

m=N 


£ U oo + 6 (1 + U M + 2P + Z M + Z oo ) for n > max (N', N"). (5) 
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Similarly 


• % 


N n 

y(n) > z(n) - u Y | h( n - m) | -f u ]T h(n-m)- 
m= 0 m= N 


Y j h(n - m) | 
m=N 


- z 


M 


N 


X N n 

mTO 



z 

oo 


n 

T h(n - m) 

x~n~N 


n 

- e y |h(n - -Vi ! for n > max (N’ f N M ) 

m^N 

> u - e (H u, , Z P 1 /. r u ) for n > max (Nh N”) . (6) 

— oo M ivi c x> 


Eqs. (5) and (6) imply 'y(n)-^ u ^ as n— > co. 

■is n — > oo 


By Theorem 2, y(n ) — * U (X) 

O. E.D. 
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SIMPLE CODING FOR K-ARY UNILATERAL CHANNELS 


D. Daetz 


The problem under consideration is related to one first treated 
by Shannon. ^ We wish to find the maximum rate at which information 
can be transmitted over a given channel with zero probability of error 
(P = probability of error). An algorithm has been obtained for gener- 
ating a code which achieves the maximum rate of information transfer 
subject to the constraints P^ = 0 and N = 2 (N is the number of letters 
in each code word) for a particular class of channels, viz. , the K-ary 
unilateral channel shown in Fig. 1. 


Since the rate R = - n ^ , where M(N) is the number of code 
words of length N in the code, when N = 2 we have R = ~ lnM(2). To 
maximize R one must maximize M(2). Let M*(N) be the maximum value 
of M(N) consistent with P^ = 0 that can be obtained with optimal coding. 


i. e . , 


M*(N) = maxM(N) 

all codes of length N9 P^ = 0. 


This work was supported by the National Science Foundation Cooperative 
Fellowship Program. 



The claim is that for the K-ary unilateral channel 


K 2 

M*(2) = K even integer 

4 


M*(2) 


K(K -1) 
4 


K odd intege 


rt 


The method for finding an optimal or suitable code (i. e. , one with N - 2, 

P =0. M(2) = M* ( 2)) is based on the equivalence of the problem of 

e 

finding a suitable code to the problem of fitting the maximum number 
of blocks of four squares (hereafter "blocks" will be equivalent to 
"blocks of four squares") without overlap onto the surface of a torus 
which is partitioned into K 2 squares (K is the number of inputs and out- 
puts of the channel). First we will demonstrate that the claimed equi- 
valence is valid and that M*(2) is in fact what we stated it to be. Then 
the algorithm will be stated and examples of its use displayed. 

Consider the K-ary unilateral channel of Fig. 1. Suppose the 

code word a. a. has been chosen for transmission. The possible outputs 
i J 

of the channel when this word is sent are 


b.b., b . b . . , 

x J i J+l 


b . . b . , 
x+1 J 


b i + iV 


^ [x] = largest integer contained in X. Hereafter, brackets will be 
used only in this connection. 
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where all indices are taken mod K. Let us construct an output table 
where the rows correspond to the outputs occurring in response to the 
first letter of the code word and the columns correspond to the outputs 
occurring in response to the second letter of the code word. Note that 
the output table is equivalent to squares on the surface of a torus because 
all indices are taken mod K. Fig. 2 shows an output table. We put 
x’s in the squares corresponding to possible outputs when a^a^ is sent. 
This has been done in Fig. 2, and it is seen that a 2x2 block of four 
squares is formed. The indices of the square in the upper left-hand 
corner of the block are the same as the indices of the letters in the 
code word sent. Hence, if we specify the row and column of the square 
in the upper left-hand corner of the block we specify the code word sent: 


i, j a i a j* ^ We ^ blocks onto the torus without overlap then we can 

always say unequivocally what word was sent given any received sequence 

b.b y One can be convinced of the need to have nonoverlapping blocks 

for = 0 by considering what happens if we allow overlap. Suppose 

that both code words a. a. and a. , .a. , , are in the code. The outputs cor- 

i j i+j j +i 

responding to a. a. are b.b., b.b.,,, b...b., b. .b. . and those correspond- 
^ 6 i J ij i J + l i+l J i+l J+l 

ing to a. + 1 a j+1 are b. + jb j+1> b. + 1 b j+2> b. +2 b j + [ . b. +2> b. +2 . Suppose the 

word b. , ,b. . , , which is common to the output table blocks of a. a. and 
i+l j+l* ^ 1 J 

a. M a, MI is received. How shall one determine what word was sent? 

1+1 j+r 

At best one can only make a guess based on the probability that a.aj ► 


b. , , b. . , and a. , , a . , , 
i+l j+l i+l j+l 


b. . , b . , , . It is clear that in such a s ituation 
i+l j+l 


P >0. Therefore, we conclude that P =0 requires the blocks to be fit 
e e ^ 
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onto the torus without overlap. 


We claimed at the beginning that 

r2 

K even integer 


M*(2) = 
M*(2) = 


K 


4 

K(K-l) 


K odd integer 


The validity of these claims can be shown as follows: 
a. K even. 

2 

When K is even we can write K = 2m, m integer. Then K = 

2 2 

(2m) = 4m is the number of squares on the surface of the torus (K rows 

and K columns in the output table). If we could use every square on the 

torus when fitting the blocks without overlap then we could place — 

4 

blocks on the torus since each block covers four squares. But there is 

a 1:1 correspondence between code words and blocks. Hence, the max- 

-2 


imum number of code words we could hope to find is 


K‘ 


M(2 ) 1 




Since the algorithm can arrange 

K 2 




Therefore, 
blocks without overlap 


on the torus, we get in fact M*(2) 
b. K odd. 

When K is odd we note that we must always have at least one 
square in each row and in each column which cannot be used since each 
block is two squares on a side and two does not divide an odd number evenly. 

Therefore, at best, we will still be unable to use K of the squares. Since 

2 2 
there are K squares on the surface of the torus we are left with only K 

K = K(K~ 1 ) squares in which to fit the blocks. Now each block uses up 

four squares so we see that M(2) £ ^ . Since the algorithm gives a 


suitable code with M(2) = 


’K(K-l) 


-Jwhen K is odd, we have M*(2) = -j. 
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The algorithm for finding a code with M*(2) code words such that the 


probability of error is zero 

tsZ 

Case I: The number of inputs K is even. M*(2) = „ One such code 

? 4 
K^ 

satisfying M(2) = is formed by using only even inputs (counting zero 

4 

2 

as even) and letting the code words be the (K/2) different two-letter 
sequences of the letters corresponding to the inputs chosen. 

Example: K = 4. See Fig. A. Use inputs "0" and "2". A suit- 
able code has code words: a Q a 0 , a Q a 2 , a 2 a Q , a-^a.,. 


0 12 3 


0 

1 

2 
3 


X 

— 


lx 



— 


— 


X 

— 


_x 



— 


— 



Fig. A: K = 4 


Case II: The number of inputs K is of the form K = 4i + 1 , i positive inte- 
ger. M*(2 ) = — U_ ^ To ac hieve a code with M*(2) code words the 

4 

only unused squares can be chosen as successive "knight's moves" (see 

Fig. B ), i. e. , if A. . denotes the jth square in the ith row, then the unused 

l > J 

squares are given by 

A j = 0,1,2, K-l 

J 

i = 2j mod K 

The square which is the upper left-hand corner of each block of four squares 
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(the squares containing "x" in Fig. B)is given by 

A. . i = 0, 1 , 2 K-l 

l 9 J 

j = (— +2i-l + — (i mod 2 ))mod K, i = 1,2,..., — — - 
2 2 4 

A set of suitable code words is then 
a^aj i = 0, 1 , 2, . . . , K- 1 

j = (— + 2i - 1 + — (i mod 2 ))mod K, i = 1 , 2, . . . , K 1 
2 2 4 

For example, for K = 9, a set of 1 8 code words formed by this rule is 

shown in Fig, B. 


0 I 2345678 

o^xi |X| I I TT] 
i x 

2 -^x x 

3X 

4 ^X X 

Cx_ III^xJ 

6 ^X X 

7X“X 

8 ^X“x 


Fig. B: K =9 


Case III: The number of inputs K is of the form K = 4 i — 1 , i positive inte- 

KL (K - l ) 

ger, M*(2) = „ Most of the unused squares can be selected as 

4 J 

successive knight's moves. More precisely, if A. . denotes the jth square 

J 

in the ith row, then the unused squares may be chosen by 


<-(¥)■ 


= o. 1.2. 


A 1 = 1,2,.. . ,K-1 

*•> J 

. _ i , K-2 .. , , . 

j - - + — — (i mod 2 ) 
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The square which is the upper left-hand corner of each block of four 
squares is given by 


A i = 0, 1,2, . . . ,K-1 
J 

1 ^ ^ (i mod 2))mod K, 


j = (- + 21 - 1 + 

J 2 2 


= 1 , 2 , 




K-3 


+ i mod 2 


A set of code words which achieves M*(2)with P = 0 is 

e 

a^ i = 0,1,2,..., K- 1 


j = (i- + 2i-l + — (i mod 2))mod K, 


i = 1, 2, 


K-3 


+ i mod 2 


An example of this case (consider K=ll) is shown in Fig, C. 



0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

0 


X 


X. 
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1 







X 


X 


X 

2 


% 

X 


X 







3 

X 






% 

X 


"x 


4 




>< 


X 






5 


X 







X 


X 

6 





X 


X 





7 

X 


X 






% 



8 






X 


X 




9 


X 


X 







X 

10 







X 


X 




Fig. C: K =11 
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ON THE NONPARAMETRIC ESTIMATION OF SHIFT 
IN THE TWO-SAMPLE PROBLEM" 


Terrence Fine 


I. INTRODUCTION 


The problem discussed in this note arises from the situation in 
which we are presented with two sets of samples, one from each of two 
independent populations of independent elements. We assume that one 
population has been shifted by an amount A with respect to the other and 
wish to estimate A without assuming more than that the population dis- 
tribution possesses a density. This problem might arise, for example, 
in designing a highly adaptive radar system; observations made without 
a transmitted pulse would establish the environmental background and 
could be compared with observations made following a transmitted pulse. 

More precisely, we are given a sample Xj, . . ., of observa- 

tions that are independent and identically distributed (i.i.d. ) as F(x) and 
a second sample of observations Y^, that are i.i.d. as G(x) 

and independent of the first sample. For some, initially unknown, shift 
A, F(x + A) = G(x)e^, where § is the class of all absolutely continuous 
distributions and G is otherwise unspecified. 

We shall present some results concerning a class of estimators 
A^ for A and discuss three members of this class having, in some measure, 
the property of robustness with respect to ^ . The basis of the criterion 
we employ to determine robustness is the asymptotic relative efficiency 
of an estimator A^ with respect to another estimator A^ , for a given 
distribution Ge defined by 



lim 

N-^oo 


var A 




var 




(i) 


The research herein was supported by the Adolph C. and Mary Sprague 
Miller Institute. 



where N = m + n and n/N— ^X. (0 < X. < 1). It is also possible to anal- 
ogously define the asymptotic absolute efficiency e— (G) by comparison 

of with the best translation invariant estimator based upon knowledge 

of G. In our usage an estimator A, is said to be robust relative to A. 

9 9 

if inf e— — (G) is not much less than unity, and it is absolutely robust 

G 4-'* 4* 

if inf e— (G) is not much less than unity. 

G 


It is known that for any finite N (=m + n) there does not exist a 
uniformly minimum variance (UMV) estimator of A. If N— >oo then a 
UMV estimator of A exists based upon the estimation of the underlying 
distribution G and is discussed by Hajek. However, Hajek's most 
robust estimator is robust only for very large samples and is known to be 
sometimes inferior to the normal scores or Hodges- Lehmann estimators 
for moderately large N (Huber). These results provide some indication 
of the nature of the problem and the results to be expected. 


II. CHOICE OF A CLASS OF ESTIMATORS 

The consideration' of possible estimators is arbitrarily restricted 
to functionals of the individual sample and combined samples empirical 
distributions. If we define the unit step function U(x) by 

f 1 if x > 0 

U(x) = | (2) 

10 if x < 0, 


with the property that 



then the empirical distributions are defined by 
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r n (x > E K tj 0 ' 51 - X i>' 

1 m 

G m (x > .2, U < X - V' 

and 

H N< X > $ X N F „( x + * + > + (' - V G m <x)> H> 

where 

^ and N = m + n. 

The class of estimators we have decided upon is given by 

p CD 

V N m J_ co K (x + V - G m (*)] + [H N { X )]dF n (x + SJ * 0, (5) 

where i|j is some nonnegative function of H^. The choice of is 

motivated below. 

Our initial approach was to adopt a metric in the space of empirical 
distribution functions and use that shift A^ which brought the shifted 
empirical distribution F n (x + A.) as close as possible, in the sense of 
the metric, to the unshifted empirical distribution G (x). The metrics 
that were considered were those of 

Kolmogorov 

sup |F (x + A) - G (x) | , 
x 111 

Renyi 

sup 

0 < a < H n < 1-a 
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and Cramer and von Mises 


r 00 _ 7 

j [F n (x + i) - G m (x)] >HH N )dH N . 


■OO 


The metrics of Kolmogorov and Renyi were discarded after a 

brief and unencouraging examination. An informal analysis of the Cramer 

and von Mises metric indicated that it did not lead to performance 

2 

superior to that of the metric given by 


W 


N 


(' 00 __ ? 

= I [F n (x+ A) - G m (x)] ^(H N )dx, 

~ OO 


and thus it too was discarded in favor of . 

2 — 2 
If we adopt , then we are to select A so as to minimize . 

2 2 

Inspection of indicates that is a piecewise linear function of 

A with at most nm vertices corresponding to shift values of -Y. + X., 

J 1 

2 

Therefore, attains a minimum at a vertex where its derivative with 

respect to A may not be defined. However, if we were to formally dif- 
2 . — 

ferentiate with respect to A and attempt to set the result equal to 

zero we would obtain something like 
00 


[F (x + A) - G (x)] i(j( H )dF (x + A) 
_oo n m N n 


+ X N / ”< F n - G m > 2 +'<H N )dF n * 0. 

- OO 


Heuristically, we might expect that for large N the second term 
is negligible in comparison with the first, and this leads us to consider 
the first term, V^, alone. We note, though, that is a piecewise 

constant function of A, or as we now write A., and we may not be able 

, — ^ - 
to find a A^ such that = 0. Thus we adopt that A^ (or one of those 


A,, if it is not unique) that brings closest to zero. This choice 

V N can now be discussed without further reference to its origin. 


of 


- 70 - 


The functional can also be expressed in terms of the ranks 

S. of the ordered X. (X } < . . . < XJ in the total ordered sample of 


X r . . . , X^ and Y, + A , , 
that 


r ' * * * CXAAVl A 1 ^ ^vj/ * • • » ^ We employ the observations 


and 


• S. - i 

FIX) = G (X. - A,) = _L 

n i n m i J/ r 


m 




1 ~ X N) + . 1 ,' X N s 

m) m i* 


( 6 ) 


to conclude that 


= 1 i 

n i=i 


V 


N 


i(l + l) - 

\n m) 


S J c 

m 


M 


■s.\ 

i 

N 


(7) 


From (7) it's easy to see that is, as claimed, piecewise continuous 


in 


the ranks S., which are implicit functions of A^, remain constant 


for variations of A that don't include the vertices -Y + X 

Y J i 


III. ASYMPTOTIC ANALYSIS OF V 


N 


We state only the results concerning the distribution of A (proofs 

qj 

may be obtained from the author). 

Lemma 1 . A sufficient condition for V N ( A^) to be a monotonically 
nondecreasing, piecewise constant function of A^ is that for 0 < t < 1 


^(t) > 


-t4>'(t) for ij;' < 0 
( 1— 1)4>*( t) for 4*' > 0. 


( 8 ) 


Lemma 2. If 4 ^ t) satisfies (8), then the distribution of A 


4 


defined by 
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- sup {a : V N (A) < 0} 


(9) 


is given by 

P(A^< A) = P(V n (A) >0). (10) 

Lemma 3. If F^(x) i s the empirical distribution for , X n 

i. i. d. as G(x - A), G (x) is the empirical distribution for Y 1t ...» Y 

m 1 m 

i. i. d. as G(x), Ge the two samples are independent, lim — = \(0 < \ < 1) 

N— >oo N 

(3K)V(0<x, y < l)^l^(x) - ^(y)l < xy f/_* . UD 

3(x 0 4 o, i)>| 44 Xq) | < oo, is defined by (5), and 

r 00 

T n = [F(x -f A ) - G(x)] 4<[XF( X + A ) + (1 - \)G(x)]dF(x+ A ) 

m ^ ^ t 


r 00 

J [F n (x + A) - G m (x)H<G)dG, 


( 12 ) 


- OO 


then A — »A as N->co=> |v m - T..I = o (N _1/2 ). 
4 1 N N 1 p ' 


Lemma 4, If 4 , satisfies ( 8 ) and (11) then 


P(/N(A - A) < A)~P(T (A+ A.) £ 0). 
^ N /N 


(13) 


r 00 

Lemma 5. If lJj satisfies (11) then J [F n (x + A) ~ G m (x)] i|>(G)dG 


-OO 


is asymptotically normally distributed with zero mean and variance cr 



given by 


<^ 2 = t N ^ N (! - X N^' 1 f 0 f S ) - tS]4>(t)iHS)dtdS. (14) 


Theorem 1. If i)j satisfies (8) and (11) then 


P(>/N(A^ - A) < A)-~erfc 


-”(A +^F) 




f* ^ 1 ^ / 2 2 

where erfc x= — r" e ^ dy, cr, is given by (14) and 

'■'x v 27 T ^ 


(15) 


r 00 

m (A) = J [F(x) - G(x - A)]i|i[\F(x) + (1 - \)G(x - A)]dF. (16) 

^ - 00 


Corollary 1. If the conditions of Theorem 1 are satisfied and, 
in addition, for small A - A 


m (A) = (A - A) J* F>(F)dF + o( A - A), 


OO 


(17) 


00 


then A^ is asymptotically normal with mean A and variance cr— given 


by 




2 = [ NX . (1 - \. T )] _1 




N N 


J F'i];(F)dFj 

xj J [min(t, S) - tS] i|4t)iWS)dtdS. 


-2 


(18) 


Theorem 1 and Corollary 1 provide the desired asymptotic analysis of 
A^ for a wide variety of 1 ) j. 
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IV. COMPARISON BETWEEN A, AND A* OF HODGES- LEHMANN 
AND A OF NORMAL SCORES 


The estimator A * defined by 


A* = med (X^ - Y.) (j = 1, 


n; i = 1, . . . , m) 


(19) 


has been proposed and examined by Hodges and Lehmann. The asymptotic 
variance of A* is known to be given by 


var A* 


[12M1 - Mn] -1 


f 


oo 


F'dF 


oo 


-2 


( 20 ) 


Lemma 6, If i|j I s identically constant then it satisfies (8) and 

(11) and if Z„ = X. - Y. (K = 1, . . . , nm), Z < Z„,. then 
K l j K K+l 


Jnm -(1 - 1 ) 
n 


. - nm ,, L . 

xi — 2 — ^ 1” rx ) 1S an in ^ e 8 er 


A , 
4> 



Z n 


is not an integer. 


( 21 ) 


Lemma 6 establishes that A, and A* are almost identical 
estimators if ip is constant. 

Theorem 2. For ^ satisfying (8) and (11), and Fe 

sup inf e*— ,, (F) = 1, and the supremum is attained for i | j identically 

tp F V 

constant. Theorem 2 establishes that in our class of estimators there 
is no estimator uniformly better than A*. Nonetheless, there exist 
estimators occasionally slightly poorer than A* but often much better, 
and we consider two such in what follows. 
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The normal scores estimators A ^ can be implicitly defined in 
terms of the statistic h(S) defined by 


h(S) = E 


V*] . 


( 22 ) 


(S) 

where w v is the S-th largest of Ni.i.d. as N(0, 1) random variables. 

If S v . . S are the ranks of X,, .... X in the combined sample of 

In I n 

Xp . . . , and Y 1 + A n , .... Y m + then the estimation procedure 

n 

is to choose A so that ^h(S.) is as close to zero as possible. The 
N i=l 1 

asymptotic variance of is known to be given by (Hodges and Lehmann 

(I960)) 

. nrv , 2 

var A n = [\(1 - X)N] _1 ]f 


oo 


F'dF 


-oo <|) [$ -1 (F)] 


(23) 


where 4> is the N(0, 1) density and 4> ^ is the inverse of the N(0, 1) 
distribution. 


Theorem 3. For Fe vJj satisfying (8) and (11) and (18) valid, 


sup inf e- (F) = 1, (24) 

F 4 / 

and the supremum is attained for ^(t) = l/(cj) [<£ *(t)] ). Theorem 3 
establishes that there is no estimator in our class uniformly better than 
A^, but there is one just as good. 
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V. 


to 


A 


PARTICULAR CHOICE OF A, 

We single out for consideration the particular 


4Xt) = < 


1 

t 

1 


1 

1-t 


for 0 < t < € 
for € < t < 1 - € 
for 1 - e < t < 1. 



corresponding 


(25) 


This choice of ip satisfies (8) and (11). 

Lemma 7. If the expansion (18) is valid, F* Q and 4 1 is given by 
(25) then 


var A^B[l2N\(l-X.)]“ l [«“ 2 +l6€+12in(l-€)] 


r o oo 

\L l 


F'itfF)dF 


(26) 


Theorem 4. Under the above conditions, 


inf e— 

F V 


= i - o(<). 


S “P % a* ( f > * 


inf e— (F) « o(e), and sup e-^ . (F) = oo. 

F V N F a fN 


(27) 


Theorem 4 indicates that for small c A, is never much worse 
than A* and sometimes infinitely better. However, we also see that 
A^ is occasionally very much worse than A^. 

We conclude that within the class of estimators we have developed 
there exist members performing as well as the Hodges- Lehmann or 
normal scores estimators but none that are uniformly better. 
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ON THE EQUIVALENCE OF FINITE -STATE SEQUENTIAL MACHINE MODELS 

Oscar H. Ibarra 


Introduction 

The equivalence of sequential machine models proposed by Moore* 

and Mealy^ referred to here as Moore machine and Mealy machine has been 
3 4 5 

shown by Cadden , Gill , and Hartmanis . The purpose of this communication 
is to give a new proof of the equivalence and to present effective procedures 
for transforming one model to the other which preserves machine minimality. 

A new upper bound on the number of states of a Moore machine equivalent to 
a Mealy machine is given and is shown to be the best possible bound. 


Notations, Terminology and Definitions 

Let Z(A) be a finite non-empty set of input (output) symbols, 
denotes the free monoid generated by Z(A) with identity /\. (called the null 

^+IA + 


L (A ) 


sequence) under the operation of concatenation. Z ' (A T ) is Z - (A - {/^ ). 

. cr n (o’, in E) and 0 < i < i < m, then .x. =cr.cr. 1 ...cr. ,(.x. =/\). 
m-li — _ _ xj l Hi u J 1 


xx. . . X* is 


If X - Q-qO^ 

We call x a tape and .x^ a subtape of x. If x is a tape in 2 
written as =2^). 

Definition 1. A Mealy machine is a system Ot = <2, S, A, f, g>, where 
L = {(r^, . . ., 0 -p} is a finite non-empty set of input symbols called the input 
alphabet. 


A = { 6q} is a finite non-empty .set of output symbols called the 

output alphabet, 

S = {sp . . . ,,s n ) is a finite non-empty set of internal states called the state set, 
f is a function (called the next state or transition function) which maps SxS into 
S; and 

g is a function (called the output function) which maps SxZ into A . 


This work was supported wholly by the Joint Services Electronics Program 
(U.S. Army, U. S. Navy and U. S. Air Force) under Grant No. AF-AFOSR -139-64. 
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A Mealy machine with n states, p input symbols, and q 
output symbols is referred to as an(n, p, q) Mealy machine. Such a 
machine can be described by a transition diagram with n vertices, 
drawn as small circles, and oriented branche s, drawn as lines 
between pairs of vertices with arrow signs pointing from one vertex 
to the other. Each vertex represents a state in S and is labeled 
the state it represents. A branch leaving state s^ and terminating 
instate s R is labeled (<r u ! 6 n )v(a l2 J 6 l2 )v . . . fij 

if f (s <r. ^ ) = s k and g (s { , <r . ^ ) = 6^ (for j = 1, 2, . . . , m ). 

(o- i j ; 6 ) ( j = 1, 2, . . . m) is called an input -output pair . Clearly, 

the branches originating from any state are labeled with the total 
number of p input-output pairs. Figure 1 is an example of a 
(4, 2, 2) Mealy machine. 


The functions f and g can be extended to mappings, 
f:Sx Z »S and g:Sx 2 by the following inductive definitions: 

f ( s,yv) = s, f ( s, x o- ) = f ( f ( s, x) , cr ) ( s in S, x in 2*, o- in 2) 
g ( s, A) =A, g(s, x o- ) = g ( s, x )g (f ( s, x ) , o-) 

The extended functions satisfy the identities: 


f ( s, xy ) = f (f ( s, x ), y ) 
g(s,xy) = g ( s, x ) g (f ( s, x ), y ) 


( s in S, x, y in 2 ) 


For s in S, x = qX^ in 2 , we may write: 

g ( s, x) = g ( s, Q x m ) = g ( s, qXj )g (f ( s, qXj ) , X x 2 ) . . . 

Definition 2 . A Moore machine is a system 
T, A, M, N>, where 2 is the input alphabet, A is the output 
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alphabet, T is the state set, M is the transition function a nd N 

is the o utput function which maps T onto B. 

As in the Mealy model, a Moore machine with n states, p 

inputs, and q outputs, briefly an (n, p, q) Moore machine can be 

represented by a transition diagram. This time, each vertex is 

labeled t; 6, where t is in T and 6 = N(t). A branch labeled 

cr. v o- 0 v . . . vcr leaves vertex t;6 and terminates in vertex t ' ; 6 ' 
i £ m 

if M(t, crp = t 1 (for j = 1, 2, . . . , m) and N(t') = 6 \ A (4, 2, 2) 

Moore machine is shown in Fig. 2. 

The function M can be extended to a mapping from Tx 2 
into T in the same way as was done for f. 

Definition 3 . Let se = < 2, T, A, M, N> be a Moore machine. 
For t in T, for any x in 2 , let h^(x) = N(M(t, x)). Note that 
for x, y in 2 ' , h t (x y) = N (M (t, xy) ) = N (M(M(t, x), y)) = 

A * 

h,.,. ,(y). Define a function h. which will map 2 into A by 

M(t, x) w ' t 

h t (x) = h t ( 0 x m ) = h t ( oXl ) h t ( 0 x 2 ). . .h t ( 0 x m ). We have for any cr in 
2 and any y in 2 + (cry) = h t (cr) ^ M / tj ^(y)* 

Definition 4. Let R be an equivalence relation on the set 
of states S of any sequential machine. R is a right congruence 
relation if and only if (s, s 1 ) in R implies that for all cr in 2, 

(f(s, cr) ^ f(s’, cr) is in R. 

Definition 5. In a Mealy machine, two states s, s' in S 
are equivalent (denoted by s = s 1 ) if and only if for all x in 2 , 
g(s, x) = g(s\ x). (Note that by definition 1, g(s,A) for all s 

in S). If there exists an x in 2 + such that g(s, x) £ g (s', x), 
we say that s and s' are distinguishable (denoted by s ^ s') and 
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r 


x is called a distinguishing sequence for s and s*. In a Moore 
machine, two states t, t* in T are equivalent (t = t' ) if and only 
if for all x in li (x ) .= h , ( x ) . If 1 (x) ^ £ (x) for some 

t I IX 

x in 2 (possibly A), t and t' are distinguishable (t t' ) and x 
is a distinguishing sequence for t and t\ Equivalence of states is 
a right congruence relation. 

Definition 6 . A machine is minimal if no two states are 
equivalent. Machine minimization techniques can be found in Moore^ 
and Gill 6 . 

Definition 7 . Let < C^= < 2 % s, A, f, g > be a Mealy machine. 

W rite g ( x ) = g ( s, x ) for any x in . Let F (Cfc) = { g ( ) | 

s s 

s in S } . F (ft.) is the set of all functions 'Oc can compute starting 
from any state s in S and is called the behavior of Vt . Two 
Mealy machines and XX. ^ are equivalent if and only if F (C^) = 

F {&.,). 

Let - < 2, T, A, M, N> be a Moore machine. We know that 
h t maps 2* into A*. Let F (Jr) = { h ( ) | t in T > . F (#-) is 
the behavior of S. Two Moore machines and ^^2 are equivalent 

if and only if F (# ) = F ) . 

In a Moore machine, ^ t (A) = h t ( A) = N(t), i. e. , the 
null sequence produces an output, whereas in a Mealy machine, 
g s <A) = g (s, A) = A • This raises the question as to how the 
equivalence of the two models should be defined. The next definition 
gives the answer. 

Definition 8 . Let be a Mealy machine and Jr be a Moore 
machine. Vt and are equivalent (denoted by VC s Jr ) if and only 
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if F (i Oi) - F (a£) , where the equality is taken to mean the restriction 
of the functions g ( ) in F [VC) and h ( ) in F («£) to mappings 

S t 

from 2 into A . Also, input tapes of the form b = x^y (x, y 
in k an integer) are not considered in comparing the two models. 

EQUIVALENCE THEOREMS 


Theorem 1 . For every Mealy machined, there exists a 
Moore machined such that 
Proof , 

Let = < 2 , S, A, f, g > be a Mealy machine. 

For every s in S, form Q , where 

s 

Q = { ( s,6)| 3 s ' * n S, 3 <r in 2 such that f ( s', <r ) = s 

s 

and g (s', o-) = 6} 

Let <*£ = < 2, T, A, M, N > , where 


T = ^ Q 

S€ S S 

For all (s,§) in Q , for any <r in 2, 

s 

M ( ( s, 6 ) , <r ) = (f ( s, or ) , g ( s, o- ) ) and N ( ( s, 6 ) ) = 6 . 

M is extended to a function from T x L into T as follows: For 

( s, 6 ) in T, X = Q x m in l- 1 , 

M((s, 6), yv) = (s, 6) 

M((S ’ 0 x m^ =(f(S ’ 0 X m^’ g(f * S ’ (^m-l*’ m-l x m^* 

It follows immediately that for (s, 6) in T, x= Q x m in L + , 

h (s, 6 ) ( 0 X m ) = N 6 )» 0 X m^ = g(£(s * 0 x m-P* m-^m*' 

Now we show that F ($) = F{<£ ). Let g ( ) be in F COt) and take 
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(# ). 


Ns, 6)< } in F {C ^ h LetX ~0 X m in 2 ' 


§s (x) = g s ( 0 X m } = g (S ’ 0 X i } g(f (S ’ 0 X 1 ) ’1 X 2 ) - • g(f (S > 0 x m-l^ m-^m* 

= h (s, 6) ( 0 X l )h (s, 6 ) ( 0 X 2 ) * ‘ ,h (s, 6) ( 0 X m ) 

A 

= h (s, 6) ( 0 x m ) 

= h( g 5)( x ) *• F (^) E F 

A , 

Let h ( ) be in F (§£) and take g g ( ) in F (Ct). Let x =Q x m in 2 . 

a a 

h (s, 6) (x)_h (s, 6) ( 0 X m ) " h (s, 6) ( 0 X l )h (s, 6 ) ( 0 X 2 ) " * h (s, 6) ( 0 X m ) 

= g(s. qXj) g(f (s, qXj), lX2 ). . . g(f(s, 0 x m _ 1 ), m . 1 x m ) 


= g (s, rt x ) 
6 ' 0 nr 


g(s, x) •• F («£) «- F (tfc). 


Q. E. D* 

Corollary L If "Ot has n states and q output symbols, & 
has n f < nq states. 

Remark. Theorem 1 provides an effective algorithm for transforming 
a Mealy machine to an equivalent Moore machine. 

Theorem 2 . The Moore machine constructed in the proof of 

Theorem 1 is minimal if is minimal. 

Proof . 

Let (s, 6 ) = (s*, 6 ') in T, (s, 6 ) ^ (s 1 , 6 f ). By definition 5 , 

A A. * A 

h (s, 6)* x)=h (s', 5 ')( x ) for a11 x in 2 • Taking x = A , h^ g 6 ^(A) 

A 

= h (s . ) 6 .)( A )<=^N( (s, 6) ) = N ( (s', 6 ') ) 6 =6 Hence, it is 

sufficient to show that (s, 6) e (s', 6), s • jt s' leads to a contradiction. 
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h (s, 6) (x) " h (s', 6) (x) 
h (s, 6) (X) = h (s', 6) (x) 


By Theorem 1, 

A 


and 


h (s, 6)< x) = «s (x) 

V. 6) (x > = « S '< X > 


g s (x) = g s' (x) 


s = s 1 ( s £ S ') 
contradiction since 


for all x in 2 


for all x in 2 


for all x in 2 


for all x in 2 


for all x in 2 


i VC 


is minimal. 

Q. E. D. 

Example. Figure 3(a) is a minimal (3, 2, 2) Mealy machine. 
Carrying out the construction given in Theorem 1, the equivalent minimal 
Moore machine is obtained and is shown in Figure 3(b). Figure 3(c) is the 
same machine with states relabeled. 

Theorem 3. For a minimal (n, p, q) Mealy machine , the 

bound n*< nq (n ! = the number of states of the minimal Moore machine 
& ,-OC) can be achieved with equality for any n if and only if p > q. 

Proof. 

Assume p < q. We will show that n' < nq (in fact, n' < np). 
The branches leaving each state in the transition diagram of i Ot, are 
labeled with the total number of p input-output pairs. Since there are 
n states, there are exactly np input-output pairs. Let q. be the number 
of input-output pairs with distinct output symbols terminating in state s., 
(note that q. is the cardinality of Q in Theorem 1). We assert that 

n Sj 

n' = q^ + q^ + . . • + ^ < np. For suppose n' > np. Then, since each input 
symbol is associated with exactly 1 output symbol, there must exist 
r > np + 1 input-output pairs. This is a contradiction. Hence, n 1 < np 
and n' < nq. 
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For p > q, let 2 = { cr^, . . . , cr^} , A = {6 j, . . . , 6 and let 

E = (<jy 6 3 ) v (tr 4 ; 6 4 ) v ... v (cr q ; 6 q ) v 6 q ) v ••• * ^p’ 6 q^ 

Consider the (n, p, q) Mealy machine shown in Fig. 4. It is easy to check 
that this machine is minimal. Since the branches terminating in each state 
are labeled with at least q input-output pairs with distinct output symbols, 
n' = nq. 

Q. E. D. 

Definition 9. Let < 2, T, A, M, N > be a Moore machine. 

Define a relation R on the state set T by: 

For t, t’ in T, (t, t 1 ) in R if and only if for any cr in 2, 

M(t, cr) = M(t\ cr ) . 

We call t and t 1 1-equivalent states . R is a right congruence relation. 

Proposition 1. Let <?& = < 2, T, A, M, N > be a Moore machine. 
t — x ^ 

If for all x in 2 , h t (x) = h t ' (x), then t and t' are 1-equivalent. 

Proof . 

Suppose there exists a cr in 2 such that M(t, cr) £ M(t', cr). 

Since & is minimal, there exists a distinguishing sequence y in 2 + 
for M( t, cr) and M(t', cr). Then .(y) £ ^Mft', ir)^ and since 

h t (cr) = h t ' (cr), we have h t (cr) ^ M(t> ff) (y) * \ M a) (y) 


A 


contradiction. 


h t ' (cry) 

Theorem 4, Let < 2, T,A, M, N > be a Moore machine. There 

exists a Mealy machine HZ such that Vt - & . Furthermore, 

IX is minimal if is minimal. 

Proof 

Let R be the relation of 1-equivalence on the state set T. 

Define a Mealy machine 'Or = <2, S, A, f, g, > , where 

S = { [ t] | t in T} , [ t] denotes the R -equivalence class containing 
t. 

f ([ t] , cr) = [ t ' ] and g ([ t] , u) = h t (cr) if M(t, cr) = t' and 
h t (<r) = N (M(t, cr) ) (t, t' in T, cr in 2). 
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Since R is a right congruence relation, the machine CM- is well 
defined. F (&&) = {h t ( ) | t in T } and F (VO) = {g|- t j | [ t] in S} 
To show that F ) c_F (Ot ), let h t ( ) be in F (c&) and take gj- ( ) 
in F ( Vt ). Let x=.x in 2 + . 

h t ( x ) = h t ( 0 x m )= h t ( 0 x l )h t { 0 x 2 ) - • ' 

= h t (qX^ h M(t, x )*l x 2^ * ‘ h M(t, 
i r / j. i i k U / r \ / 


v v n r q 

= g([ t] . q x j) g([ M (t, qXj)] , jX 2 ). . . g([ M(t, 0 x m l )J , 

= g([ t] . o x l) s( f ([ t] . o^), jX 2 )* • • > 0 X m-l*’ m-l X m^ 

= g [ t] *0 
g [ t] 


x ,) m-1 
m-r v 

i x ) 

m-1 m' 


(n x ^) 


(*) 


F (<£) C F («fc). 


Now let g|- ( ) be in F (#) and take ( ) in F 

in S + . 


Let 


x= x 
0 m 


g [ *] W = 8 [ »] ( 0 x m> = g( I *1 ' 0 X 1> g<f([ tJ ' 0 X 1>' l x 2>‘ 

“8([ > g x j) q x i) ] * i x 2^' 

• • • g([ M(t, n x ,) ] , iX , 

& ' L x * 0 m-r J m-1 m) 


h t ( 0 X l )h M(t, Q x ) ( l x 2 )# # * 
h M(t, 0 x m l r m-l x m^ 

h t ( 0 X 2 ) '" h t ( 0 X m ) 

= h (_x ) 

t 0 m' 

= K(x) /. F(0t)cF («£). 


If o(y is minimal, we assert that is minimal. For suppose 

[ t] = [ t 1 ] in S, [ t] t [ t'] . Then g([ t] , x) = g([ t * ] , x) for all x 
in S + =^> h^(x) = h^ 1 (x) for all x in Z + '=^>(t, t 1 ) in R (by Proposition l)r^ 
[ t] = [ t 1 ] ~ contradiction. 


Q. E. D. 

Remark. Theorem 4 provides an effective procedure for transforming 
a Moore machine to an equivalent Mealy machine. 

Example*Figures 5(a) - (c) illustrate the procedure for transforming a 
Moore machine to an equivalent Mealy machine. 
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Fig. 3(a). A minimal (3, 2, 2) Mealy- 
machine T% r 
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Fig. 3(c). Moore machine of Fig. 3(b) 
with states relabeled. 
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2, 2) Moore machine 






Fig. 5(b). Minimal (4, 2, 2) Mealy machine 

({t., t.} is an R -equivalence class). 
^ J 
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(I;0) 



Fig. 5(c). Moore machine of Fig. 5(b) with states relabeled 
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A NOTE J&N THE PERMANENT OF A MATRIX 


Jean-Paul Jacob 



The permanent of a matrix, redefined below, is not a new 
concept. Surprisingly, however, it seems that one of its most trivial 
applications, namely, the counting of non-zero additive terms in the 
expansion of the determinant of a matrix, has not yet been used. Our 
main result using this fact is presented in reference 2. 


Definition: Given an nx n matrix A = fa. .1, the permanent 

f t - i,J J y 

of A , denoted A | is by definition 



a l,P(l) a 2,P(2)- 


a n,P(n) 


where the sum is extended over all possible permutations P of the 
integers 1, 2, . . . , n . 

Hence, the permanent of a matrix A is the same sum as the 
determinant of A with all terms prefixed by the positive sign. 

Lemma 1 : Given a square matrix A of order n, the number 

of non-zero summands in the expansion of its determinant is equal to 
the permanent of A, where A is obtained by substituting 1 for all 
non-zero entries of A. 

The term "summand" and the proof of lemma 1 can be found in 
reference 2. 


This work was supported wholly by the Joint Services Electronics 
Programs (U. S. Army, U. S. Navy and U. S. Air Force) under Grant 
No. AF-AFOSR -139-64. 
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Example 1: Given A below, how many summands are there 

in the expression of | A | ? 

a 1 3 
a 23 
a 3 3 


= 1 + 1 + 1 = 3 


One can trivially verify that, indeed, 

I A | = a i i a 22 a 33 ' a 13 a 22 a 31 " a ll a 23 a 32 

The counting character of the permanent leads to a natural 
application, i.e. , solving problems in combinatorial analysis. Examples 
2 and 3 illustrate this 

Example 2: Five gentlemen wearing hats go to a banquet. When 

handling their hats at the checking counter, the receptionist forgets to 
give them tickets. At the exit of the banquet the receptionist gives 
randomly one hat to each of the five gentlemen. WTiich is the probability 
p that no gentleman will get his own hat back? 

Solution : Consider the 5th order matrix below. 


A - 


L 11 


22 


a 31 a 32 


The answer is : 


■+- ■+ 

I A I 


1 0 1 
0 1 1 
1 1 1 
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a ll 

a 12 

» • • 

a 1 5 

a 2 1 

a 22 

• • • 

a 25 

a 51 

a 52 


a 55 


Each additive term in the expansion of A^- is of the form: 


a l,P(l) a 2,P(2) 


•5,P(5) 


( 1 ) 


th 

If we associate with the first subscript of a. the i 

^ i, P(i) 

gentleman and with the second subscript the gentleman who gets the 
th 

i gentleman's hat, there is a 1-1 correspondence between each 
term in the expansion of | A ^ | and each possible way in which the 
receptionist can distribute the 5 hats. 

On the other hand, the cases in which at least one gentleman 
receives his own hat correspond to terms like (1) in which there is at 
least one index i, i < 5, such that 


i = P(i), 


which implies a^ to a diagonal entry. In the numerator of p 

(the number of "favorable cases") we do not want to count such 


cases. 


Therefore : 
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0 1111 
10 111 
110 11 
1110 1 
11110 


11111 

11111 

11111 

11111 

11111 


In Theorem 1 below we will show how permanents, like those 
above can easily be computed. Theorem 2 gives an elegant solution 
for the numerator of p.l 

Example 3^: In how many ways N can 8 rooks be placed on a 

chessboard so that none can take another and that none stands on the 
white diagonal? 


I i 

Answer: N = |Ag - Ig| 

Where Ag is the 8X8 square matrix with all its entries equal to 1 
and Ig is the 8X8 identity matrix. N is the permanent of the 8X8 
matrix in which all diagonal entries are zero and all other are 1 (see 
conjecture 1) 

Again, the answer was obtained by observing the 1-1 corre- 
spondence between the additive terms in the expansion of the determinant 
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0 


a l 2 a l 8 

a 21 0 a 28 

a 81 a 82 ’ * ' 0 

and the cases described in our problem. 

Lemma 2 : The permanent of a matrix A remains unchanged 

if one interchanges any two rows (any two columns) of A. 

The proof of lemma 2 is trivial from the definition of perman- 
ent. 


Theorem 1 : Let A be an n X n matrix whose entries are either 
"1 M or M 0". If there is at most 1 zero in each row and in each column 
of A , then 


w =£ ( -‘> k (!) <"- k » 

k=0 

where Z is the total number of zero entries in A. (note that, from 
the assumptions, Z < n) 

The proof of Theorem 1 goes by induction on Z. Also, from 
lemma 2 and the assumptions of Theorem 1, we lose no generality in 
assuming that all zero entries of A are in its main diagonal, namely, 

a l, r a 2, 2’ • • * * a Z,Z * 
a) Z = 0 

Then the determinant of A has as many terms as the possible 
permutations of n objects, i. e. , n! 
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n ! 


l i -‘» 

k=0 


k 0 


(n -k) ! 


b) Assume that Theorem 1 is true for all 0 < Z < p - 1 . To prove that 
it is true for p, notice that to obtain 


-h 


from 


+ 


"+ 


0 

a 1 2 

« til i • « 

!p 

a In 

a 2 1 

0 

• 3. ^ • • 

2p 

* a 2n 

V 

a p2 

0 

. a 

pn 

a nl 

a n2 • ' 

. a 

np 

. a 

nn 

0 

a l 2 

• 3.-. • • 

x p 

' a ln 

a 2 1 

0 

• ^ • • 

2p 

• a 2n 

V 

a p2 - 

a 

PP 

a 

pn 

a nl 

a n2 ‘ ‘ 

a 

np 

a 

nn 




( 2 ) 


(3) 
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we have to substract from (3) the number of non-zero summands which 

contain a . A little reasoning shows that this number is 
PP 

/ j \ 

(n-l)! - (P; 1 ) (n - 2) ! + ^- 2 j (n-3)! + ... + ( n - p) ! 
which can also be written as 


I 

k= 1 


(- 1 ) 


k+1 


(p-M 
\k-l / 


(n -k)! 


(4) 


Using the induction hypothesis to compute (3) and subtracting 
(4) from it, our answer becomes 


M) k ( P ^ ! (n-k)! (-l) k+1 ( j (n-k)! 

k=0 k=l V ' 


P-1 


k= 1 


x . / 

= n! + 2^ ( - 1)k ^k 1 ; (n-k)! 


«♦£> 

k=i 


D k P-»; (n-k)! = 
\ K " 1 / 


P-1 

n! + (_l) k (n-k)! 

k= 1 


\ / 


ip- 1 ; , ! p-i 
\ k ' + k-1 


\ f 


+ (-1) P (n -p)! = 


>-l 


= n! + V 


^ (-1) (n-k)! { P j + (-l)P(n-p)! = 


k=l 


= J(- 1)k (P'| (n-k)! 


k=0 


Q. E. D. 


r 
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A particular case of Theorem 1 is given for Z = n. This is 
the case of Example 3, for instance, in which 


8! 

8! 

8! 

8! 8! 

8! 8! 

— 

“ + 

- — + — 


2! 

3! 

4! 

5! 6! 

7! 8! 


( 5 ) 


Considering that 8! = 403Z0 and 4! = 24, the exact value of 
N above cannot be computed with a slide rule. Instead, we may try 
to use ingenuity. Rewrite (5) as 



Observe now that between the brackets in (6) are the 7 first 
non-zero terms in the expansion of e ^ . Since our answer N has to 
be a positive integer , one may suspect that the 7 terms above 
approximate e~ well enough for N to be the closest integer to (8! )e ^ 
A search of this type originated our 

\ 

Conjecture 1 : Let A be an n x n matrix whose entries are 

"l" or "0" being zero iff it is a diagonal entry. 


4 + | I I 

Then | A | = (n! ) e | 

i. J 

where the square brackets in the right-hand side mean "closest integer 
Notice that conjecture 1 is, in fact, a theorem, but its proof 
would involve steps of direct verification and convergence properties 
of the sequence 


k! 

k=2 
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The answer to example 3 is then 


r 


N = 


40 320 e' 1 


14 853 


Final Remark: During the course of our divagations on the 

utilities of the concept of the permanent we found many other applica- 
tions. Their importance was, however, minute or nil because easier 
alternative methods were known. The most sophisticated of such 
applications is in graph theory, where one is interested in counting 
the total number of trees in a graph. In the case of the graph of a 

network, the sum of all tree admittance products is the determinant 

4 

of the nodal admittance matrix. There is a 1-1 correspondence 

between each tree of the graph and each non-zero subdeterminant (of 

order k) of a reduced incidence matrix A (of order kxe) of the graph. 

r 

If we compute adding (e - k) rows of ones to it and compute - in a 

special way - the permanent of this new matrix, we will obtain the 

total number of trees in the graph multiplied by (e - k)! . By " special 

computation 11 we mean: compute the permanent of the complete matrix 

2 

x by taking a Laplace expansion along the (e - k) rows of ones and 
computing modulo 2 the subdeterminants of the original k rows. 
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THE NUMBER OF TERMS IN THE GENERAL GAIN FORMULAS 
FOR COATES AND MASON SIGNAL-FLOW-GRAPHS 


Jean Paul Jacob 


This communication is intended to answer questions of the sort: 
given a Coates flow-graph, 2 or a Mason signal-flow-graph, 3, 4 ’ 5 how 
many "connection paths," or "loop products," will one have to find in 
order to compute numerator and denominator of the general gain 
formula ? 

We initially recall the definition of the determinant of an nx n 
matrix A = [a^ as being 


I— I S (S1§n P) ^ a 2> p ( 2) • * ' a n, 


P(n) 


5T— 11 

= 2_ ( si 8 n p ) TT a 


i=l 


i. p (i) 


where the summation extends over all possible permutations P of the 
integers 1, 2, ..., n. 

Easier than the determinant is the concept of the permanent of 
a matrix A = [a. .] which is, by definition,^* the same sum as the 

1 9 J 

determinant of A with all terms prefixed by the positive sign, i. e., 


+ + 


permanent of A= A = Ya. a- -d/o,... a 

- 1 ^ !» P(l) 2, P( 2) n. 


P(n) 




This work was supported wholly by the Joint Services Electronics 
Frogram (U. S. Army, U. S. Navy and U. S. Air Force) under Grant 
No. AF-AFOSR-139-64. 



Example 1: Find the permanent of A below 


1 1 1 
0 1 0 
1 0 1 
0 1 1 


r 


- 

h 


1 1 0 


1 1 1 

:r 

1 1 0 

+ 

1 0 1 


0 1 1 


0 1 1 


* (1 + 1) + (1 + 1 + 1) = 5. 


Notice that we obtained the permanent above by using Laplace's expansion 
along row 2, with all signs taken positive. It is not difficult to verify 
that the Laplace expansion technique is valid for permanents. 

Definition 1: The simplified connection matrix C of a Coates flow-graph 

2 

C Q (graph G with source node 0 deleted) is a square matrix 

in which the (i, j) entry is 1 or 0, being 1 iff there is, in G Q , 
an edge from node j to node i: it is understood that the entry 

(i, i) is 1 iff there is a self-loop at node i. All nodes of Gq 
are represented in C. v 

Example 2: Find the simplified connection matrix of the Coates 

flow-graph of Fig. 1. 


0 < 




Fig. 1 


-no- 


1 


2 


3 


4 



1 1 
0 1 
1 0 
0 1 


1 0 
0 1 
1 0 
1 1 


Note that, from the definition above, the simplified connection 

matrix can also be trivially derived from the conventional connection 
7 

matrix. Change all nonzero entries of the latter for 1. 

Below we refer to "summand" as an additive term of a sum 
containing no parentheses. For instance, in 


a+b 

d 


c 


e 


= (a+b)e-cd = ae+be-cd 


there are 3 summands. 

Lemma 1: Given a square matrix A, the number of nonzero summands 

in the expansion of its determinant is equal to the permanent of 

/m 

A, where A is obtained by substituting 1 for all nonzero 
entries of A. 

v- n 

Proof: |A| = 2_(sign P)TTa . (1) 

P i=l l » U ' 


+__+ n 

l?l =Im 

P i=l 


P(i) 


( 2 ) 


By comparing (1) and (2), we see that there is a one-to-one 

correspondence between the nonzero summands of | A | and the nonzero 

+ + + + 

summands of p£|. But each of the nonzero summands of |2C( is 1, 
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and we are adding as many l ! s as the nonzero summands of |A|. 

Hence the lemma. 

Lemma 2: The number of summands in the denominator A of the 

general gain formula for a Coates graph G is equal to the 
permanent of the simplified connection matrix C of Gq. 

Sketch of Proof: Each additive term of A is called a connection gain 

C (Gq). In a very similar way to the proof of Lemma 1 of [2] , 
one can prove that there is a 1-1 correspondence between 
the connection gains and the nonzero terms of |c|. By 
Lemma 1, the number of nonzero terms in the expansion of |c| 
+ + 

is exactly | C | . 

+ + 

Example 3: For the graph of Fig. 1, there are |Cj| connections, and 

+ + 

from Example 1, |Cj| = 5. The 5 connections of this graph are 
presented in Fig. 4. (page 117). 

Example 4: How many connections are there in the graph of Fig. 2? 



Figure 5 exhibits these 4 connections (page 11S). 
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Lemma 3: The number of one-connections from source node 0 to 

*f .+ 

node j in a graph G is given by |C J | where C J is obtained 
from the simplified connection matrix of when column j 
is substituted by a new column 0; an entry (i, 0) in column 0 
is 1 iff there is an edge from node 0 to node i. 

Illustration of Lemma 3: In Fig. 1 the number of one-connections 
from node 0 to node 3 is given by 

+1204++ ++ 

1110 110 I 1 

0 10 1=110 + 1 
1 0 1 0 0 0 1 0 

0 10 1 

Sketch of Proof: Once more [2] must be cited: Lemma 3 follows from 

Lemma 2 if one observes the 1-1 correspondence between the 

2 

one -connections of Fig. 6(a) and the connections of Fig. 6(b) 
of [2]. 

Example 5; Consider again the example of Fig. 2. How many one- 
connections are there from node 0 to node 3? 

+ 

1 1 

1 0 =1 + 1 + 1 + 1= 4. 

1 1 

Figure 6 shows these 4 connections (page 119), 

Definition 2: The simplified connection matrix C of a Mason signal 

flow-graph Gq is a square matrix, the (i, j)-th entry of which, 
i 4 j, is 1 or 0, being 1 iff there is, in Gq, a connection 




+ 

1 1 

0 1 =2 + 2=4. 

1 0 
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from node j to node i. The (i, i)-th entry is either 1 or 2, 
being 1 if node i has no self-loop. Once more the index 
i (or j) runs through all nodes representing variables. 

Note that, from the definition above, the simplified connection matrix 

7 

can be trivially derived from the conventional connection matrix. 

For a same graph, the off-diagonal entries of C are unique, 
independently of whether the graph is a Coates or a Mason flow-graph. 
The diagonal entries of C differ by 1, being higher for the Mason 
flow-graph. 

Example 6: Suppose Fig. 1 represents a Mason flow-graph. Then 



Lemma 4: The total number of terms in the denominator A of the 

general gain formula for a Mason signal flow graph is equal to 
the permanent of its simplified connection matrix. 

Illustration of Lemma 4: For the Mason graph illustrated in Fig. 1, 
there are 


+ + + + + + 


2 

1 

1 

0 


2 

1 

0 


2 

1 

1 

0 

2 

0 

1 

= 2 

1 

2 

0 

-f 

1 

0 

2 

1 

0 

2 

0 


0 

1 

2 


0 

1 

1 

0 

1 

1 

2 










= 2(8 + 2) + (1 + 4 + 1) = 26 


terms in the denominator of the general gain formula. 
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/ 


Sketch of Proof: If a system of n equations in n unknowns x^, x^» 
.... x n is given in matrix form Cx = then a Mason signal 
flow-graph is a graphical representation of this system when 
rewritten as 


Cx = U- C')x = Cq 


with certain rules used to represent the equations above. 

I is the n yn identity matrix. (I - C') differs from C only 

in that it has one more summand in all diagonal entries. 

5 

There is a 1-1 correspondence between the summands in 
the expansion |l - C'| and the summands of A, Apply Lemma 1 to 
the matrix (I - C') and notice that (I - C') is exactly the simplified 
connection matrix of the Mason signal flow- graph. 

Lemma 5: The number of terms contained in the expansion of A^ 0 -^, 

where A, ^ is the A associated with the k-th forward path 

K k 

from node 0 to j , is equal to the permanent of C , where 

C is the matrix obtained from the simplified connection matrix 

by deleting all rows and columns corresponding to nodes in the 

k-th path. 


Illustration of Lemma 5: Consider the Mason signal flow-graph illustrated 

in Fig. 3. Its simplified connection matrix is; 

h 



Fig. 3 


v 
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c = 


2 1 
1 1 
0 1 
0 0 


0 

1 

1 

1 


1 

2 

3 

4 


We are interested in x 4 /x Q . How many terms has the numerator of 
x 4 /x 0 ? With path ab (which touches nodes 3 and 4) we have 


1 2 1 + 
1 1 


= 3 terms. With path ceb (which touches nodes 2, 3 and 4) 


+ + 


w 


e have 2 = 2 terms. The denominator (Lemma 1) has 


2 

1 

0 

0 


2 

1 

0 

1 

1 

1 

0 

= 

1 

1 

1 

0 

1 

1 

0 


0 

1 

1 

0 

0 

1 

1 






= 2 + 2 + 1= 5 terms 


, , , x 4 ab(l - h - £g) + ceb(l - h) 

Indeed, as one can verify, — = i - ( h + {/l dc) + ha ' c ~ • 

Proof of Lemma 5: Lemma 5 is a trivial consequence of Lemma 4 

because Aj^ is just the A for a graph obtained from G by 
deleting all nodes (and incident connections) which belong to 
path k. 
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I 



Fig. 4. The 5 connections of the flow-graph shown in 
Figure 1. 
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Fig. 5. The 4 connections of the flow -graph shown in 
Figure 2. 
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ON THE NUMBER OF ROOTS OF A REAL POLYNOMIAl^ 
INSIDE (OR OUTSIDE) THE UNIT CIRCLE USING 
THE DETERMINANT METHOD 



E. I. Jury 

In this note, a simple procedure is presented which gives the 
number of roots outside and inside the unit circle of a real polynomial 
F(z). It requires the evaluation of either odd or even determinants 
depending on "n" (the degree of the polynomial), i. e. , whether n is 
even or odd. The form of these determinants is presented elsewhere* 
and we present only the results: 


n - even 

The number of roots outside the unit circle (if none of the 
appropriate A^ + vanishes) is given by the number of sign changes 
of the following determinants: 


A 1 
A i 

+ B j , 



A 3 

+ B^, 

A 1 

’ n-1 

+ B’ . 
n-1 

A 1 
A 1 

+ B[, 

A i 

- B i’ 

A 3 

- B^, 

A n-1 

n-1 


A 'n-l 

- B' 

n-1 ’ ‘ 

• • J 

A i - 

B i, 



A- + 

B i 

J • 

• • i 

a ;.i 

+ b ;.i 



plus one or zero depending on the sign of F(l) F(-l). 


The research herein was supported by the Air Force Office of Scientific 
Research under Grant AF -AFOSR -292-64. 
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n - odd 




> or, 

J 


plus one or zero depending on the sign of F(l) F(-l). 

The number of roots of F(z) = 0 inside the unit circle is n minus 
the number of the outside roots, provided F(l) F(-l) 4 0. 

Example 

Let n = 10, and using the following value 
A j + B ^ < 0, A^+B^<0, Ag + <0, A^+B^O, A^ + B^>0 

A^ + B^<0, A ^ - B j < 0, A^-B^>0, A^-B^>0, 

Ay - By < 0, A'y -B' 9 <0 

and F(l) F(-l) < 0, we obtain the number of changes of sign of the 
determinants as 3. Since n = 10, and F(l) F(-l) < 0, the number of 
roots outside the unit circle is odd, i. e. , three- The number inside 
the unit circle is 10-3 = 7. 

Example 

To find the root distribution of the following polynomial 

F(z) = 3 - 2z - ^ + z^, n = 3 

2 


i, A!, + B^t A' 4 +B' 4 , .... a;_j + b;.j 


1, ai, - bi,, A4 - B4. .... a;_j - b;_! 


A' . - B' A' - B!,, 1, 

n-1 n - 1 £ c 


A 2 + B 2 A i,-1 - B n-1 
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In this case : 


«r 


* 


A' + B! 




21 

2 


< 0 


Hence , 


1 > 0 , < 0 
1 > 0, A' z - B' z < 0 


There are at least two roots outside the unit circle. Since F(l) F(-l) > 0 
(no single root can exist inside the unit circle), hence the third root is 
outside the unit circle. 

The singular cases are discussed in detail in references (1) and 

(2). The proof of the above simplified counting is based on certain 

2 1 
manipulations of the following formula : 


2<A t 2 - B^ 2 ) = (A> 


B k-1 


)(A k+l + B k+l) 


+ (A k-l + B k-l)( A k + l - B k + l) 
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STABILITY OF SINGLE-LOOP FEEDBACK SYSTEMS 
C. T. Lee and C. A. Desoer 


We consider the systems shown in Fig. 1: it is a single-input 

single-output single-loop feedback system; N is a memoryless time- 
invariant nonlinear element and G is a nonanticipative time-varying 
linear system. The purpose of this paper is to extend the sufficient 
conditions for stability given by Popov 1 and Desoer 2, 3 . 


+ 

Ua 

e 

V m 

N 

a 

G 


- s 

/ — 







Fig. 1. The System S 


Of course, since G is time-varying there is no frequency domain inter- 
pretation. However it is of interest to note that the linear system must 
satisfy some very simple assumptions, 

N. 1. The nonlinear element is characterized by 

<*(t) = <p[ e (t )] (1 ) 

where the nonlinear characteristic ^ is a piecewise continuous function 
defined on (- oo, oo), i. e. , <p may only have a finite number of discontin- 
uities in any finite interval, and at any such discontinuity, e. g. , e^, 
both ) and ^(e^+ ) are defined and finite. Furthermore, 

N. 2. There are two positive constants £ and k such that 

0 < £ ^ ^ k - £ for all e ^ 0 (2 ) 

The research herein was supported by National Aeronautics and Space 
Administration under Grant NsG-354 (S-2). 
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and 


*( 0 ) = 0 . 


G. 1. The linear subsystem G is characterized by its input-output 


equation 


' i 

y(t)=z(t)+ ^ g(t,t 2 )a(t 2 )dt 2 t > 0 


( 3 ) 


where z(* ) is the zero-input response which depends on the state of G 
at time 0. 

In other words, G is a nonantic ipative linear time-varying 
system represented by a superposition integral and N is a time-invariant 
memoryless nonlinearity whose characteristic lies in the sector by the 
line a = te and a = (k-b)e. The equation for the feedback system may 
be written in the operator form as follows: 


y = z + G a, 


(4) 


where a is given in terms of y by (1). To state the main result we must 
list more assumptions: 

G. 2. g(t 1 ,t 2 ) = 0 whenever t^ < t 2 

|g(V t 2 )| < g M < 00 for all t L and t 2 > 0 
- 00 

] 0 |g( t 1 » t 2 )| dt 2 1 M for all tj > 0 

g (t, t^ ) 0 as t oo uniformly for t ^ in any finite interval (0, T ). 

2 

G. 3. For all initial states, z(0) is finite, z(* ) and z(* ) are in L (0, oo), 


z (t ) 0 as t oo. 


- 126 - 



G, 4. The solution of (4) is assumed to satisfy the following requirement: 
for any finite t 

y( * ) £ L 2 (0,t) and y( • )£ L 2 (0,t). 

In other words, G. 2 requires that the impulse response be bounded 
for all times, that the response to an impulse applied at any time t^ goes 
to zero, that the zero state response of G to any bounded input be bounded. 
G. 3 requires that the response due to any initial state be square integrable 
as well as its derivative, and that it goes to zero as t -» oo. G. 4 may be 
thought of as a condition eliminating certain kinds of finite escape time 
response. In fact, if z(«) is bounded it is easy to show that y is bounded 
by an exponential; see the proof of Assertion I in reference 3. The 
main result is stated in the form of a theorem. 

Theorem 

Consider the system S shown in Fig, 1. Let u - 0. Suppose 
that the assumptions N. 1-2, G. 1-4 are satisfied. Under these assump- 
tions, if there are two positive numbers q and 6 such that 

< x, (G + qDG + ^)x> > 6<x, x> for all xc L 2 (0, oo ) , (5) 

2 

then the output of y of S is bounded on [0, oo), belongs to L (0, oo) and 
goes to zero as t-* oo, whatever might be the state of G at time 0. Here 
the operator D denotes the time derivative operator, i. e. , 

Dx = ^(x(t)). 

Before proving this theorem, we shall prove two auxiliary 
lemmas. 

Lemma 1 

Let A be an operator (not necessarily linear) mapping from 
Hilbert space H into itself and let x^, i = 1,2, 3 be points in H such that 
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Xj = + A(x^ ). 


( 6 ) 


Under the above assumptions, if there exists 6>0 such that for all x € H 


<A(x),x> > 6<x, x>, 


(7) 


then 


£46 <X 2’ X 2 X 


Proof 
F rom ( 6 ), 

<x^ x 3 > = <x 2 + A(x 3 ), x^> = <x 2 , x 3 > + <A(x 3 ), x 3 >. 

F rom (7), 

<Xj, x^> > <x 2 , x^> + 6 <x 3 , x^>. 

Now , 

<x 2 ’ x 3> + 6 <x 3' x 3 > = <^ x 3 + 2?T X 2’ N ^ X 3 + tJsT X 2 >_ T6 <X 2 ,X 2 > ‘ 

Since the first term of the right hand side of the equality is nonnegative, 


<x r x y* = <x 2’ x 3 > + 6<x y V > - Tr <x 2' x 2 > 


' <x rVtTT <x 2 'V' 


This is the desired inequality. 
Lemma 2 

Let x« L 2 (0, oo)nL°°(0, 00 ) 


Q.E. D. 


r> oc 

A |8< t r t 2 , l dt 2 - m2 


for all t^ and t 2 >0 
< 00 for all t^ > 0 


* Zames defines the operators satisfying ( 7 ) " 


6-passive operators" 
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g(t, ) -* 0 as t -* oo uniformly in any finite interval [ 0, T], 


= fgft. 


Then w(t)= \ g (t, )x (t 2 )dt 2 goes to zero as t oo. 


Proof 


w 


(t *l £ |gtt.t 2 )| |x(t 2 )|dt 2 |g(t,t 2 )|dt 

•( f _ig(t.t 2 )iix(t 2 )i 2 dt 2 y /2 < m ('j t | g (t,t 7 )iix(t 2 )i 2 dt ; ,y /2 


■J 


Because x £ L (0, oo), for arbitrary € > 0 we can choose T sufficiently 
large so that 


poo 

i. I X M' 


dt 2 < € V 


Then, since g(t, t^ ) -* 0 as t -*• oc uniformly, for arbitrary > 0 there 
is T ' so that 


f | g (t, t ) | dt 
Jq * 


2 < e 2 for t — T'. 


Henc e , 


t 'p 

l*< t » t 2 ) l l x(t 2 , ! 2dt 2 = Isft'Vl + jV (t * t 2 , l l x(t 2 ) l 

Since x e L°°, there exists a finite number such that |x(t)| < x^, t > 0 


'dt. 


I 0 I g (t, . t 2 ) I I x(t 2 ) 1 2 dt 2 < J T | g i (t, t 2 ) I dt 2 + g M I x(t 2 ) I : 


'dt. 


i X M € 2 + «M e l fort > T ’ 


I w (t ) I 1 M < x m € 2 + g M € l ,l/2 


for t > T' 
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w (t ) -*■ 0 as t -* oo. 


O.E. D. 


Proof of Theorem 

Let a subscript T placed on any function, such as y, denote the 
function defined by 

ry(t) 0 < t < T 

y T (t) = 

t.0 elsewhere 


From (4) 

y T = z T + Ga T for 0 < t < T. 


Let us differentiate the above equation and add. We get 


y T + qV T + 


T Ic T 


= z„ 


+ qz j + (G + qDG 


From G. 3, G. 4, N. 2 and the assumption of the theorem, it is implied 
that the assumptions of Lemma 1 hold; hence, for any T > 0 


T 1 

- jy(t) + qV(t) + £ a(t)|a(t)dt <i- § |*(t) + (t)| 2 dt. 


Hence, since u = 0, 



e(t) = -y(t). and a(t) = <»[e(t)] 

r T i 

<p[ e(t)] dt + q J <*[ e (t )] e (t )dt <-^ 



z (t ) + qz(t)| at = C 


The constant C is finite and is independent of T. Call and the two 
terms of the left hand side. By (2), > 0, hence, Jg C 



( 9 ) 


. ( 8 ) 
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Since z (0 ) is finite by G. 3 and e(0) = -y(0) = -z(0), the constant C' equal 
to the right-hand side of (9) is finite. Now by (2 ) we get 

| e ( T )| 2 <|£l for all T > 0. 

Therefore e ( • ), ^ 2 ’ ^- n dy(-)are bo unde don [0, 00 ). Using (9) in (8) and 
N. 2 we get 

J i£ c ' (10) 

Now by N. 2 again 

^ j e (t) ?[e(t)jj. ?[e(t)]dt >^ 2 J |e(t)| 2 dt for all T > 0 . 

2 

It follows that e (• ) € L ( 0 , 00), and the same is true for y(-). Thus, 

00 2 2 

y(- )« L flL and a( • )e L x riL . By Lemma 2, (Ga)(t) — 0 as t -* 00 . 

Finally, with z - 0 by (G. 3), (4) implies that y(t) - 0 as t - 00 . Thus 

we have shown that ye L^flL 00 and -* 0 as t -*■ 00 . Q. E. D. 

It might seem that the conclusions are restricted to statements 
concerning the zero-input response of the closed loop system, but this 
is not the case. Suppose that u t- 0 and that it satisfies the condition 
G. 3, then (4) may be rewritten as 

-e = -u + z + G a with e = u - y. 

Since u - z in the present case satisfies the same conditions as z in the 
statement of the theorem, we therefore have the corollary: 

Corollary 

Let the input u satisfy the conditions G. 3, and let all the other 
assumptions of the theorem hold. Then the output y of S is bounded, 
on [ 0, 00 ), belongs to L (0, 00 ) and goes to zero as t -* 00 , whatever might 
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be the initial state of G at time 0. 

It is obvious that the results can be extended when input-output 
relations are represented in a vector form. 

Example 


G 



Fig. 2. 


N satisfies N. 1 and N. 2. Assume | m(t) | < p < oo for t > 0. 

It can easily be shown that 

f ”^ t l" t 2^ P t l 1 

g( t l» t 2 ) = | e J m(T)dT |l(t 1 -t 2 ) 

l t 2 J 

and g(t^, t 2 ) satisfies G. 1, G. 2. By direct calculation, G. 3 is easily 

shown to hold; G. 4 follows by the Gronwall- Bellman lemma (5) applied 
to (3) where (2) is taken into account. In the following we shall find a 
stability region for k. 
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(Ga)(t) = fe<t T ) m (T } r e ^ a(£) d£dr 


= e 


1 f e T m( t) f e ^<*(£)d£dT 
‘-'n Ja 


(DGo)(t) = -(Ga)(t) + m(t) C e ^ ^ a{£)d£ 


((G + DG)a)(t) = m(t) J V (t '^a(£)d£ 


<a > (g + DG) tt > i < h y°°iof(t) i i<*(£) i d£dt 


0 "0 


< \± < a, a > 


< a, (g + DG + i j ff > >(i - < a, a > . 

Therefore, if ^ - }j. > 0, then the conclusions of the theorem hold for the 
system under consideration. 
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A CONSTRUCTIVE DERIVATION OF THE CAPACITY 
OF A BANDLIMITED CHANNEL 

D. J. Sakrison and L. P. Seidman 


L Introduction 

The expression 

C = Wlo S2< 1+ 2!fv> 

o 

for the capacity of a channel limited to bandwidth 2W in the presence 
of additive Gaussian white noise of two sided spectral density N is 
well known ([ 1], Chapt. 5). However, the proof usually given of the 
positive side of Shannon's coding theorem for this channel is not con- 
structive [2]. That is, it does not demonstrate for a rate less than C 
the existence of a code with large but finite delay that can achieve some 
desired small error probability. The major defect of such proofs in 
this regard is their use of the sin x/x functions for signalling. To 
make the proof constructive, one needs to replace these functions with 
suitable functions whose duration is equal to the block length of the code 
(one half the delay). The suitable functions for use in this regard turn 
out to be the Fourier transforms of the prolate spheroidal functions 
discussed by Slepian, Landau and Pollack [3, 4, 5, 6]. 

Consider a channel over which we are allowed to send signals 
of time duration T; each signal to have energy less than PT and energy 


This work was supported wholly by the Joint Services Electronics 
Program (U. S. Army, U. S. Navy and U. S. Air Force) under Grant 
No. AF-AFOSR- 1 39-64. 
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outside a band + W c.p. s. less than tPT, 0 < - <1. This channel 
has additive white Gaussian noise of two sided spectral density N . 
We shall show in the next section that the capacity of such a channel 
is 


c = Wlog 2 <l + l°g 2 e 

0 0 

by showing that for large WT this channel can be reduced to a collec- 
tion of amplitude continuous, time discrete channels whose joint capacity 
is easily computed. Since there exist constructive proofs of the coding 
theorem for such a collection of time discrete channels, our argument 
will be constructive. 

II. Solution 

Taking the Fourier transform of the functions discussed in [ 3] , 

we have a set of time limited functions <p.( t) i = 0, I, . . . , which are 

2 1 

complete in L [0,T] and orthonormal ([6], p. 70). Further, there 
exist constants such that > 0, V strictly decreasing with i, and 
is the energy of c in the frequency band |f| < W. 

Slepian has shown ([5], p. 11) that if b is fixed and 

i = 2WT + bln - 2WT 
2 


then 


lim - 

1 .2 


WT-** 1 + e 


Ti b 


Now consider a channel with white additive Gaussian noise of 

il density N . 

1 o 

and (3) are satisfied: 


spectral density N q . We are to send a signal S(t), such that (1), (2) 


S(t) = 0 for t not in [0»T] 


(1) 
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energy S(t) < P 


( 2 ) 


{energy of S(t) out of band |f| <W}<eP (3) 

Then representing the signal in the basis of prolate spheroidal wave 
functions we have, 

00 

S(t) = iU Sk * k(t) 

and the constraints are 


00 



k=l 


The problem is to choose the S k , satisfying (4) and (5) to obtain channel 
capacity. 

Arbitrarily choosing d Q , let 
N x = 2WT - d Q fn(| 2WT) 

N ? = 2WT + d f n( — 2WT) 

4 o 2 

Our channel is then represented by three distinct and statistically 
independent channels; 

chan 1 — {Sj, • • • » 1 

chan 2 = {S-, , • • • , S.. } 

N l+1 1N 2 
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clian 3 { ^tsj * ^tvj * • • • » } 

iN Z+l iN 2+2 


and we must maximize the channel capacity 

c = c^Pj) + c 2 (p 2 ) + c 3 (p 3 ) 

th 

where Ck(P.) is the capacity of the i channel with energy P. used 
on it, subject to the constraints; 

P 1 + P 2 + P 3< P 

2 

|s k l d-V < «p 

Suppose P = P, + P-, < P, and P. = aP . Then, as is easily shown, 
rr o 1 c. — 1 o 

the optimum value of a is 


2W, 


N, 


a = 


2W 


Nj 2WT - d Q fn(^ 2WT) 
x + 2W 2 n 1 +(N 2 -N 1 ) N 2 2WT + d Q in(^ 2WT) 


so a -+ 1 as 2WT -* oo. Thus no power is used in C 2 . Now for k > N^> 


1 - V. ^€,^1 

k — 1 — 


1 + e 


-d TT 
o 


for k > N 9 , 1 - V. < 1 - € ~ < 1 — 

2 k - 2 — . , 2 


1 - e 


+ d TT L 

o 


where and e 2 may be made arbitrarily small by choosing d Q large. 
Thus, we must have for large 2WT 
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■< 


t 



k=0 


(1 - OP 


Z l s k| 2 = P 2 = 0 
k=N,+l 


oo 



k=N 2 +l 


and then the capacity of the channel is (from the usual formulas) 


C = C[(1-«)P] + C 3 [eP] 


= W log 2 



(1-OpV 6P 

2WN 2N 

o j o 


log 2 e. 


If e -*■ 0, we have 

c = w ^2 (> + mw; ) 

which is the formula obtained by considering the channel as one with 
2WT coefficients. It must be observed, that while the above conclusions 
are strictly true only as WT -► oo, agreement is quite good for modest 
WT, ([1], p. 4). 
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STABILITY ANALYSIS OF MONOTONE 
FEEDBACK SHIFT REGISTERS* 

C. J. Tan and A. Gill 

In a recent paper Massey and Liu^ introduced a special class of 

feedback shift registers, or FSR^s, called monotone. The stability of 

this class of FSR’s was examined in that paper. However, Massey and 

Liu were unable to formulate a general stability test for monotone 

FSR'So In this note, a general test (which is also the necessary and 

sufficient condition) for the stability of monotone FSR's is derived, by 

2 

the use of Boolean matrices. 

Description of autonomous FSR's by Boolean matrice s . 

A general autonomous FSR consists of delayers and logical 
elements arranged in the form as shown in Fig. 1. 



The research herein was supported by the Air Force Office of Scientific 
Research under Grant AF-AFOSR-639-64. 



i 


is the binary output symbol of the i-th delayer, and 

f i ( s 1 , s 2 sj is an arbitrary Boolean input function for the 

i-th delayer. The binary n-tuple s = (s^, s^, ... , S n ) is the state 
of the FSR. 

A Boolean matrix is, in effect, a Karnaugh map of Boolean 
functions in its row space. For example, consider the FSR shown in 
Fig. 2. 



The feedback function f^ and f^ can be expressed as the sum of 
all minimal polynomials, i.e., 

= = (1 ^ s^)\J (1 f\ s i f\ s 2 ) U (0 f\ s^C\ s^) U (1 f\ Si f\ s^) 

f 2 = s l = <on 5 2 >u a n S]L n s 2 ) U(on s n s 2 ) u a a s n s 2 ) 

The matrix equation characterizing this FSR is B 2 s = f, i.e., 


1 

1 

0 

l" 


S 1 

= 

i 

l 

1 

o 

1 

0 

1 


1 

03 

r\j 

l 

if 

_ f 2_ 


If each state s^ = (s^, s 2> ... , s n ) of an FSR with n delayers 
is regarded as a binary number with the least significant digit at the 
left, or equivalently at the top row in the column matrix s, then each 
column in B^ can be represented by its decimal equivalent state 
vector, e.g., for above example, 
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9 



Furthermore, in a Boolean matrix B , representing an FSR 
with n delayers, the state vector at the i-th column, starting from the 
left, is the successor state of state i-1. Hence, for the above example, 
state (0 0) is followed by state (1 0) , state (1 0) is followed by state 
(1 1) , etc. 

The identity Boolean matrix A is one for which f = A s = s . It 
is the matrix which expresses the set of equations 1 = s., i = 1, ... , n; 


, A^ 


0 10 1 


0 0 11 


0 10 10 10 1 


0 0 0 0 1 1 1 1 


In general, expressed in decimal equivalent, 


A 
— n 



n-1 n-1 n-1 

2-1 2 2+1 



( 3 ) 


Monotone FSR's . 

An FSR is called monotone if and only if W(B s) ^W(s) , for all 
s_, where W(sJ denotes the Hamming weight of the state s. So, an 
FSR is monotome if the Hamming weights of its states in every state 
sequence form a monotonically non-increasing sequence. 

For any FSR of the type shown in Fig. 1, the state function 
l = ( f l> f 2 , ••• , f n ) is equal to (f^ Sj, s 2> ... , s ). Hence, all 
rows, except the first row, of the corresponding Boolean matrix B are 
fixed; and row i, where 2 — i— n, is the same as the (i - 1) st row of 
the identity matrix A . That is: 
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x 

X, 

X-, 

X, 


x 'V 1 

XI 1 - 1 

X X n 



0 

1 

2 

3 


2-1 

2 

2+1 2-1 



0 

1 

0 

1 


1 

0 

1 ... 1 


B = 

0 

0 

1 

1 


1 

0 

0 ... 1 


— n 







. 




0 

0 

0 

0 


1 

0 

0 ... 1 



0 

1 

0 

1 


1 

0 

1 ... 1 



0 

0 

1 

1 

. . . 

1 

0 

0 ... 1 


A = 







. 



— n 

• 






• 




0 

0 

0 

0 


1 

0 

0 ... 1 



0 

0 

0 

0 

• • 0 

0 

1 

1 1* • 1 



Each one of the 

2 n positions on 

the top row can be filled with 

either 

0 or 

1 . 

Each different 

combination represents one of the i 

? n 

2 Z 


possible distinct feedback functions f^. For a monotone FSR, W(B s) fEW(sJ. 
Hence = • • • = = 0. 

If it is further required that X^ n “-*-= ... = X^ 11 ^ 3 ! > then the row 

space of is the cyclic shift of the row space of the corresponding 

identity matrix A^, as can be seen from equation (4) . The feedback 

function of this particular Boolean matrix is f^ = s n • Such an FSR is 

3 

called a pure cycling register . Each successor state is a cyclic shift 
of the previous state. Hence all states fall in a cycle which consists 
of all states such that i is a binary shift modulo 2-1 of a binary 
number j such that j is not contained in some other cycle. For 
example, for n = 3 
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0 

0 

1 


0 

1 

1 




0 0 
0 1 
_0 0 


1 1 1 
0 1 0 
0 0 1 




1~ 

1 

1__ 


and every state is in one of the 4 cycles shown in Fig. 




If , say, is filled with 0 instead of 1, then the successor 
state of (Oil) will not be a cyclic shift of 6 but rather a state belonging 
to another cycle with weight one less than the original cycle; in this 


case B 

[?] . 

■ o ■ 
0 


i J 

.1. 


Stability , 

A monotone FSR is called stable if and only if s = 0 for every 

state of the FSR, and some finite positive integer N. In other words, 

there cannot be any cycles except the trivial one in the state diagram. 

n 

Let the set of states S = { 0, 1, 2, • • • , 2-1} be the index set 
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S can be decomposed into disjoint subsets, each containing all the 
states in CL, where is one of the cycles arising from the pure 
cycling register. Hence, 


H 0 = {0} 

H j = ( 1, 2, 4, ... , 2 n_1 } 
h 3 = { 3, 6, 12, • ■ • , 3x2 n_1 } 


H 2?l 

Let X = { Xq, X^, • • • , be the symbols at the respective 

columns at the first row of .B . They fall into subsets according 
to their respective indices; that is, 

G 0 = { V 

°1 = <X 1 ' X 2 > ' X 2 n ' I > 

°2-l = { X Z?1> 

Theorem : A monotone FSR is stable if and only if at least one 

n 1 

of the X. ’s , for i^ 2 , in each G in (6), is equal to zero. 

s 

Proof: Let h_ be a subset of H_ , for every H_ in (5), 

s s s 

such that h = {i : X. = 0, 2 n . Hence for any k € h , B k = L, 
where H^ and W(H^) = W(H g ) -1 . Given any _i e H gl B^ i = k, for 

some h g , and some positive integer L. Therefore B(B^ i) = 

B^ k = f_€ H • Hence, given any initial state except zero, its weight will 



decrease after some finite number of transitions until it reaches 
state (). Since B 0 = 0, this monotone FSR is stable. On the other 
hand, if the monotone FSR is stable, then there cannot be any cycles in 
the state diagram, and every state has to reach the zero state. There- 
fore, at least one of the X for i22 n_1 , in each G has to be zero. Q.E.D. 

~ s 

From (6) it is obvious that X^n-land X^ are the only elements 

with indices larger or equal to 2 ~ l in their respective subsets 

and G.£ . Therefore it follows from the above theorem that 

X n_ l- X n =0 
2 2-1 U - 

4 

Golomb and Welch have shown that the number of cycles obtained 
from the pure cycling register is 


Z(n) = L E * (d) 2 n/d 
d |n 

where 4* (d) is Euler's 4> -function , and summation is over all divisors 
d of n . Since in a Boolean matrix of a monotone FSR we have 
■^0 = Xi = • • • = X^n-l- 0 , there are exactly 2 n ^ digits which are not 
fixed. However, at least Z(n) - 1 digits, one in each G. , except G 

l c 0 

in (6), have to be zero. Therefore, the following corollary is immediate: 

Corollary: There are at least 2 2 -Z(n)+1 distinct sta bi e 

n-stage monotone FSR's. 

A table of Z(n) , for n up to 10, is given in page 3, Ref. 3. 
Example: Consider the FSR of Fig. 4. 
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f l = s 3 S 2 S 1 


Fig. 4 


The Boolean matrix for this FSR is 


^3 = 


0000 0010 

0101 0101 

0011 0011. 


G o = { x o > 

G 1 = {X r X 2 , X 4 ) 

G 3 = { X 3’ X ! V 
G 7 = { x 7 } 

x 4 = x 5 = x 7 = 0 


This monotone FSR is stable. 


The state diagram of the FSR is shown in Fig. 5. 


* 

4 


« 
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^ t 

CODING GRAPHS AND INFORMATION LOSSLESS AUTOMATA T 


P. P. Varaiya 


Let (jt be a finite, directed graph with vertices v € V and 
labeled arcs satisfying the following conditions: each arc has a label 
cr « 2 attached to it and each label may be used any number of times. 
There are no restrictions on the number of arcs leaving a vertex, ex- 
cept that there is at least one arc having each vertex (i. e. , there are 
no terminating vertices). A graph satisfying these conditions is called 
a coding graph. '*’ 

Let (v, v 1 ) be ah ordered pair of vertices (not necessarily dis- 
tmct). We will define three subsets of 2 as follows: * 

x = (cr^, . . . , cr n ) e Fq (v, v‘) if there is at least one path from 
v to v 1 with (cr^, . . . , <x n ) as the labels on each of these paths, in the 

order in which they take place on this path. 

x = (cr^, . . . , o" n ) e F^ (v, v 1 ) if there is exactly one path from 

v to v' with (cr^, o" ^ j . . . , cr^) as the labels on that path 

x = (c r^, . . . , cr n ) e F 2 (v, v 1 ) if there are at least two paths 

from v to v 1 with (cr^, . . . , <r ) as the labels on each of these paths. 


* + . 

2 is the free semi-group, under concatenation, of all non-empty 
finite sequences of elements from 2. 


t 


The research herein was supported by the National Aeronautics and 
Space Administration under Grant NsG-354 (S-l). 
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Clearly F Q (v, v') = F^v, v') (J F^v, v') and F^v, v') F 2 (v, v') = <f> 

We shall say that F^(v, v') is the set of tapes of 2 + which is information 

lossless with respect to (v, v'). The reason for this term is that given 

(v, v') and (<r^, . . . , e F^(v, v 1 ), we can determine the path from v 

to v' with which that sequence is associated, i. e. , (tr, ) preserves 

1 n r 

the information of the path from which it is derived* However, if 

<r n ) t F 2 (v,v'), this information is lost. 

We will say that G is information-lossless with respect to 
(v, v') if F 2 (v, v ') = 4> • We will prove the following theorem. 

Theorem: The sets F q(v, v 1 ), F^(v, v 1 ) and F 2 (v, v 1 ) are regular 

subsets of 2 + . 

2 

The term "regular subset" is defined as follows: 

Definition: A nondeterministic finite automaton (n d f a) is a system 

S, M, A, F > where 2, S are finite nonempty sets, A 
and F are subsets of S, and hi is a map from 2)(S into the power 
set (set of all subsets) of S. 

We will say that a tape (cr.^, .... ir^) e 2"*” is accepted by 
if there is a sequence (s Q , s i» • • • . s n ) of elements from S such that 

a) s Q € A, s n e F 

b) s. +1 « M(<r i+1 ,s.) 0 < i < n-1. 

Let T (i OC/) - , <r n ) € 2 + | (tr^, cr n ) is accepted by 

Definition: A subset TC 2 + is regular iff T = T (OV) for some n d f a. 


The theorem will be proved by a sequence of a simple results. 

Pact L F 0 (v, v') is regular. 

Proof: Consider the n d f a '&0 = < 2 , V, M, {v} , {v'} > where 

M ( (r . v.) = {v. € v | 3 an arc in G from v. to v. with label <r } . 
J 1 J 
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Then ( cr^ o" n ) € T (£/Q iff 3 a sequence (v Q , . . . , v ) with 

v 0 = v ’ v n = v '’ and ^ an arc from v. to v. +1 with label <r 

for 0 -i-n-1; iff («r cr ) e F (v, v 1 ). 

x n u Q. E. 

Now for n — 0, consider the relations ^ V * V defined 

below: 


p o 

= {(Vj, v-) 

K • 

v} 



P 1 

= {(Vj. V ( ) 

1 3 (vj, 

-) . 

p o ;3 

£ 2; 3 two distinct 


arcs fr 

om v. 

l 

to V . 
J 

and from 

v. to y with the same 


label cr 


P n+1 ^ V j’ V £^ 13 ( v i> 6 P n l 3 o' € 2; 3 arcs (not 

necessarily distinct) from v. to v. and from v, to v 

1 J ' k i 

with the same label cr 
Fact 2: For n — 1. 

P n - ^ V j’ v f ^ I 3 f V’ 3 (o^, . , . , o" n ) f - £ + ; 3 two paths 

from v. to w and from v. to such that they have 

the same labels (o^, . . . , cr n ) and such that the first arcs on 
these paths are distinct. 

Pr oof: Follows from the definitions. 


Pact 3: a > P n = * P n+1 = <!> 

b > P n * P „ + k £or k > P „ +j+rk = P n+j Vx 

V o - j < k 

c) 3 N - 2 1 ^ ^Vk>N,3 n — N such that P 

k 

Proof: Follows from the definition of P and the observation that 

n 


> 


0 . 


= p . 

n 


P n ~ V ^ V ^ n - 


Q.E.D. 
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Let P = 


P = P . 

i n i n 

n=l n=l 


Let (v^, v. ) be a fixed pair in V V. Con- 


struct an n. d. f. a. (w, v.) as follows: 

( v i» v j) = <2, P Q P, {(v.,v.)> , {(Vj,Vj)}>. The function M 
is defined below: 

a ) If (vp v^,) = (v^, v^ ), then for cr c 2 , 

M (cr, (v , v^)) = { (v^ 1 , v ^' ) I two distinct arcs from to v^' 
and from v^ to v^' with the same label cr } . 

b) If (v^.v^) =£ (v^v^, and (vj.v^) « P, then for <r t 2 , 

M (<r, ( y i» v 2 )) = {( v i'» v ?') I arcs (not necessarily distinct) 
from to Vj' and from to v^ 1 with the same cr } . 

c) Otherwise, let M (cr, (v^, ^ = ^ • L et 


U(v. , v . ) 
i J 


def 


T ( (v.,v.)) 


that 


Lemma 1: U(v^, v.) is the set of all tapes ( cr ^ , .... u n ) « 2 such 

there exist two paths from v. to v. with the same labels (cr,, . . . , cr ) 

i j I n 

and such that the first arcs in these paths are distinct. 

Proof: Suppose (cr^, . . . , o" n ) e U(v.., v^). Then a sequence 

<(Zo’^o’ (v l' v 'l ) ’---’ ( V v n^ suc ^ ^at 
(X 0 ’l'o )= ^i’V’ ( y n ’^n' )= ( v j. v j) and 

<Xi+i« x'i+i) € M ^i+i’ 0 < i < n-1 . 

In particular (v^, v'^ ) c M(cr^, (v^,v^)) so that two distinct arcs with 

the same label cr . from v. to v. and from v. to v, 1 . . ” . (v. , v', ) c P, 

1 i — 1 i — 1 — 1 — 1' 1 


P. 


By induction we can show that (v., v.') « P. P i and that arcs from 

—i — i i 

v. to v. +1 and from v'. to vh+j with the same label <r^ + ^ (trp . . . , <r n ) 

satisfies the required condition. 

Conversely suppose that (o-j, . . . , cr n ) satisfies the given condition. 
Then ((^q* y 'o^’ • • • » (X n » y ' n )) such that 



(v , v' ) = (V., v'.); (v , v' = (v., V.". • * 

'—o’ — o' ' i r n’ — n ' j j 

3 two distinct arcs from v. to v, and from v. to v 1 . with 

1-4 l—l 


the same label cr^. 


3 arcs from v. to v._^ and from v 1 . to with the same 

label cr. , . 

l+l 

It is clear then that (v., v 1 .) e P. — P, i > 0 and that 

l — i' l 

(V i+1 , V'. +1 ) € M ^i + 1> (Xi> Z.'j) ) 1 = 0 n ~l 

. *. (o-j cr n ) e U(v i; v.) Q. E. D. 

J 

We now return to the proof of the theorem. 


Suppose (o-p .... cr n ) e F^ (v, v'). Then 3 two distinct paths from 
v to v' with the same labels (o^, . . . , cr n ). Since the paths are distinct, 
there must be an integer 

i, 0 — i — n-1, such that the paths differ for the first time in their (i+l)st 

arc. It is clear then that 

if i > 1, (cr, , . . . , <r.) « F (v, v.) for some v.e v 
1 l o ' j j 

and (o\ tr ) e U(v., v’). 

l+l n' j 


i = 0, (dp . . . , tr n ) e U (v, v'). 

In other words, 

F 2 (v,v') - U (v,v') UU V £ v F 0 (v, v.) U (v., v')* 

J 

Conversely, if (a^, . . . , e U (v, v'), certainly (a^, <r n ) e F 2 (v, v'), 
and if (ir^, . . . , <r.) e F Q (v, v^) and (o-j +1 , . . . , <r n ) € U (v^, v ')> 
then (o-p .... <r n ) e F 2 (v, v'). 

Lemma 2 : For (v, v') fixed, F^ (v, v 1 ) is regular. 

Proof: F.,(v, v') = u(v, v') U U F (v, v.) U(v., v') 

v. € v ° J J 
J 

If A, B are subsets of AB = { xy |xe A, ye B} , AB is called the 
product of A and B. 
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Now the sets U(v, v'), F q (v, v.), and U(v., v 1 ) are regular. Since 
the class of regular sets is closed under finite unions and finite products 
(Ref 2), F ^ (v, v 1 ) is regular. 

Proof of the Theorem: We have shown that F q (v, v') and F^v, v') are 
regular. Since F^(v, v 1 ) = F q (v, v') - F^fv, v 1 ) and since regular sets are 
closed under set differences (Ref. 2), it follows that Fj (v, v') is regular 

Q.E.D. 
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THE SARDINAS AND PATTERSON TEST '' 

P. P. Varaiya 


Let 2 and A be two nonempty finite sets. Let 2 + and A + denote 
the free semigroups generated by 2 and A, respectively, i.e. , 

Z (A ) is the free semigroup of all nonempty finite sequences from 
2(A) under concatenation. Let 2 - A + be an arbitrary one-one map 

from 2 into A . Let <}>: 2 + - A + be the natural homomorphic extension 
of <t> to 2 defined inductively by: 

A 

<(>(< r) = c(> ( cr ) for all cr € 2 

and 

«t>(xcr ) = <j)(x)4> (o- ) for all x « 2 + (re 2. 

Definition. We say that <j> is a code iff $ is one-one, i. e. , iff $ is an 
isomorphism of 2 into A + 

Sardinas and Patterson 1 were the first to formulate a test to 
determine whether or not a given <j> was a code. Their proof, however, 
is not completely satisfactory and leaves some questions unanswered. 

Our proof of their result will answer all these questions. 

Our derivation is partly based on an incorrect proof by 
Schiitzenberger . With the help of a convenient notation due to 

Schutzenberger, we formulate the S. and P. test. 

Definition. Let (S, • ) be an arbitrary semigroup and A, B be (possibly 
empty) subsets of S. Then 

a "In def 

A B =SBBB {s€S|3a«A such that a- s e B} 

= {s<S| A'sflB ^ <f> } 

and 

BA {scS |3aeA such that s- a e B} 

— — { s € S j s- AflB i <)> 

The research herein was supported by the National Aeronautics and Space 
Administration under Grant NsG-354 (S-l). H 
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Theorem (Sardinas and Patterson): Let 2, Abe nonempty finite sets 

1 * "-j- 

and let 2 , A be the free semigroups generated by 2 and A , respec 

+ /V 

tively. Let <|>: 2 -+ A be a one-one map and let <|> be the natural homo 
morphic extension of <f> to 2 


Let P q = <{> (2 ) - A and define , 

p. = p _1 p 

1 o o 

P , = P _1 P UP _1 P for n > 1. 

n+1 o n n o — 


Then <j> is a code P OP = <f>V/n > 1. 

n o T v — 


Proof : 

I. 


Notation: Let x, y eA . We say that 


II. 


x <y M3 z € A + 3 zx = y 

W e note that x<y , y <w =£ x < w. 

Next, for n > 1, we define sets Q - A + as follows: 

— n 

q(n ) € O n iff q(n ) satisfies conditions (i) - (vi) below: 


( i ) 3 p L € P q for 1< i n + 1. 


(ii) 3 n’ with 1 < n* < n such that 


(UU P,P 2 ...p n ,q(n)= p n , +1 ...p n+1 

(iv) p l * p n' + 1 
(v) q(n ) < P n + 1 and 

(vi) Vi > O.VJ >0 P 1 + i . . . P n ,q(n) t p n , +j . . • P n+1 

III. We shall prove by induction that P = Q Vn > 1. 

n n — 

a) p i = Cy. By definition of P p q(l) € Pj iff 3pj € P , 3 P 2 « P Q 
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b) 


such that p 1 q(l) = p 2 iff q(l) € Q 

Assume that P = O . We shall show that P . = Q ... 

n n n+1 n+1 

Let q(n+l) € P^ . = P _1 P UP^P . Then 3 p « P , 
n+1 o n n o K o o 

3q(n)c P = O such that either p q(n+l) = q(n) or 
n n l o 

q(n)q(n+l) = p . In the first case, by condition (iii)we have, 
p r . . P n ,P o q(n+l) = p r . . p n ,q(n) = P n , +1 . . . P n+r 

Clearly, q(n+l) satisfies conditions (i) - (iv) and (vi) since q(n) 
satisfies those conditions. Condition (v) is satisfied because 
q(n-fl) < q(n) and q(n) < p n+1 , so that q(n+l)c Q n+r 
In the second case, using (iii) again, 

P r • • P n ,q(n)q(n+1) = p n , +r . . P n+1 q(n+l) = P r . . P n ,P G . 

It is easy to verify that conditions (i ) - (vi) are satisfied so 
that q(n+l)e Q 

Conversely, let q(n+l ) € Q n+ ^» Then, 

P : - • • P n .q(n+1) = P n . +1 - • • P n+2 where (i) - (vi) are 
satisfied. 

By (v)and (vi ), we must have 
either q(n+l) c p n+2 < p n ,q(n+l) 
or 

q(n+l) < p n ,q(n+l) < P n+2 * 

In the first case let q(n)q(n+2) = P n+2 * Then, 

P n'+1' • * p n+l q * n * = p l* * ’ P n'* 

It is clear that conditions (i) - (iv) and (vi) are satisfied by q(n) 

since q(n+l) satisfies them. Since P n+2 = q(n)q(n+l) < P n , <q(n+l 

we have q(n) < p , so that (v) holds and hence q(n)c Q = P . 

n L n n 

Since q(n)q(n+l) = P n+2 € we have q(n+l)e P P q - P^ + ^. 

In the second case, let q(n) = p n ,q(n+l) < P n+2 - Then^ 
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P r • • p n'-l q(n) = p n'+l’ • • p n+2’ 


Surely, q(n) satisfies (i) - (vi ) so that q(n)c O n = P . Since 

q( n ) " p„,q(n + l) we have q(n+l)« P _1 P ? P ... 

n on n+I 

[V. Proof of the theorem: 

A. To prove that the condition is necessary, let q(n)c P HP = Q HQ. 


Then 3 p. e P , 1< i < n+1 such that 
1 o — — 


non 


«V • ■ p n' q<n) = p n'+l’ ' ' p n+l and P 1 * P n'+1' 

Let p. = 4> ( cr ) and q(n) = <j>(cr ) with cr., <r « H . Then, 

11 q i q 

<r, t cr , and 
1 n+1 

A A A 

4> (c Ty . . . , o* n ,, 0 -^) = 4> (cr n , +1 , . . . , cr n+1 ) so that <j> is not 1-1. 

B. To prove that the condition is sufficient, suppose $ is not one-one. 
Then3(cr^, . . . , c r^) t (cr^’, . . . , <r|) with 


<X<T V ..., O-J = p r • . p n = • • • - *;> = Pi- . . pV 

Let i be the integer such that = pj j < i and p^ + ^ ^ p! + ^, 

,hen p i+i- • • p „ = p l+r • ■ p m and p i+l * p'l+r 

Now let j and k be the smallest positive integers such that 


P i+1 - • • P i+ j ~ P^- • • P^+j. It is easy to see that 
either P i+j or p! +k belongs to 


Q. E.D. 


Remarks : 


1. The test is valid where 2 and A are countably infinite sets. 

2. If we define, in a dual manner, 


-1 


P, = P P’ 1 and P . = P P' 1 U P P" 1 
1 o o n+1 o n n o 
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we obtain, by using a dual proof that 

<)> is a code <=>P n P = <b Vn> 1. 

i. Suppose that 2 is finite and let P = 4>(2). 

■f ^ 

Let P = {x € A |3p € P such that x«p } . Then P is finite 
1 o r o J 

and that P = Q £ P. From this observation we obtain the 
n n 

following corollary. 

Corollary 1: 

a) P=<|>=^P_=<b 

n T n+1 Y 

b) If P = P for some n and k > o, 

n n+k 9 

then p n +jk+r = P n+r V^° V°l r<k - 

c) 3 n < oo such thatVn > N 3 m < N such that P = P . 

— n m 

Proof: a) and b) follow from the definition of the P ! s and do not 

n 

require finiteness of 2. 

c) Let | P | = cardinality of P. Then since jP | < oo the 
set R of all subwords of words in P is finite. Let N = z\^\ + 1. 

Then, since each P^ *= R there exist integer s t , k < N such that 
P^ = P^. c) then follows from b). 

Corollary 2: Let 2 be finite and let cj>: 2 be an arbitrary map. 

A i 

Let cj) be the homomorphic extension of cj) to 2 . For n > o let 

2 = (J 2 l . Then3 N < oo such that 

n . 

Kn 


cj) is a code < > <\>: 2 

Proof: 11 = = > M is obvious. 

1 1 > 1 1 . 


A^ is one-one. 


N 


Suppose <j> is not a code. Then3n < oo such that 
q(n ) c P^ P| P . By Corollary 1 c ) we can suppose that n < N, It is 
clear from our proof that 3 (p^ . . . , P R+1 ) * P Q such that 

P l'“ P n'* {n) = P n '+1-*- P n+l withp l * P n'+L 
Let <f) ( cr^ ) = and tj> { cr^ ) = q(n). Then cr^ ^n'+i and 

* ((r l V’V = * (o n'+l’’-*’ 0 n+l ) 

Since n < N, the assertion is proved. O. E. D. 

Definition: Let xe 2~^ and let x = (o^, ...» ^ n )< We will say that x 


has length n and denote it by |x . 
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Definition: Let x and y belong to 2 + (A + ). We say that 

x oC y 44 3 ze S + (A + ) such that xz = y 

and x == y 44 xocy or x = y. 

Definition: Let cj> be a code. Let $ be the extension of <)>. By the 

delay of <j> we mean the smallest integer l > o such that 

X&c S+, Vyc Z+,Ve € Z + with |a|>i* 

OC OC 

<|)(xz) = 4>(y) =>■ x = y . 

If there is no such finite integer, we say that 4> has infinite 
delay and write 4 = oo. 

Corollary: Let be a code, and m be tbe smallest integer such that 

P is empty. Then m < 4+1. Also, 4 = oo iff P is nonempty Vm. 
m — m 

Proof: A: M 4+l < m. M 

If 4 = o the assertion is true because m > L 

Suppose 4 > o and suppose that 4+1 > m. ThenBx = (cr^, . . . , cr^ ) 
z = ( cr ,,, . . . , <r , ) with s > i , 3y = (<r' , . . . , <r ' ) such that 

^(xz) = $(y) and 

x ^ y i. e. , (o-j, . . . , cr m ) =£ , or'. ) j. 

Since $ is 1-1 (2) implies that 

A A 

<t> (x z) oC <t> (y). 

Without loss of generality, we may assume that 
=£ tr'^ and 

. . . , tr' n _ 1 ) (cr^, . . . , or r> ••• , ' r r+s^ 

4> (o-’p • • • . <r' n ). 

Now let 4> ( cr t ) = p^ and 4> ( cr 1 ^ ) = pV. Then (3) can be written as 

p'r • • P ’n-1 * p r • • p r+s * p 'r • • p 'n with p i * p 'r 
Now let qt be such that 

Pi- "■'r+s' 1 = P’r ••P'n' 

If | z | = o, then x z = zx = x x« 2^\ 
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We claim that qe O , , , = P 

H r+s+n-1 r+s+n-1* 

Clearly q satisfies conditions (i) - (v) given in the proof of the theorem. 
Suppose J i > o, rij > o such that 


But then 


3 l+i-’-Pr+s 
P r 


’ q = P 'l + j--- p, n- 

.p 1 .. Since cj) is a code, this means that 


i ^ r 1 j 

(°"l» . . . , o% ) - ( cr . . . , <r' j ) so that = cr'^, which implies p^ = p' 

which is a contradiction 


q € P 


r+s+n-1 


m > r + s+n-1 > 1+i+l-l = i+1 


B. "m = oo =9 £ = oo " 

Suppose P^ is nonempty for all m. Let n be any fixed inte- 
ger. We will show that £ > n. 

Let q(m) € = P m . Then, 3p^P Q 1 < i < m+s such 

that 

Pr--Pm'P (m) = P m-+r*-Pm+l and P 1 ^ P m -+1 

It is clear that by taking m sufficiently large we can make m' > n+1. 

Let <)> (<r. ) = p V i, and choose x = a-,, z = cr . tr , and 
11 1 c m 

y = . . <r m+1 . Then | z | = m'-2 > n and 

<t>(xz) :\ 4> (y) but x y since =£ <r l+ .. Q. E. D. 
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SHADOWS OF FUZZY SETS * 

L. A. Zadeh 

The concept of a fuzzy set and some of its implications were 
discussed in a recent report. 1 In this note, we shall focus our atten- 
tion on the properties of shadows of fuzzy sets. 

In order to make our discussion self-contained, we shall begin 
by recapitulating several concepts relating to fuzzy sets which will be 
needed in the sequel. 

1. Roughly speaking, a fuzzy set is a "class" of objects in 
which there may be grades of membership intermediate between full 
membership and non -membership. Thus, a fuzzy set A in a space 

X = {x} is characterized by a membership function p A which associates 
with each point x in X a real number p A (x) in the interval [0,1], with 
K A (x) representing the grade of membership of x in A . 

2. A fuzzy set A is said to be contained in a fuzzy set B, 
written as ACB, if and only if p A (x) < p B (x) for all x in X . 

3. The union of two fuzzy sets A and B is denoted by A U B. 

If C = AUB, then by definition H c (x) = Max [p A (x), P B (x)] , xeX, 
meaning that the grade of membership of x in AUB is the larger of 
the grades of membership of x in A and B. 

4. Similarly, the intersection of two fuzzy sets A and B is 
denoted by aOb. If C = ADb, then by definition p c (x) = Min[p A (x), p B (x)] , 
X € X . 


The research herein was supported by National Aeronautics and 
Space Administration under Grant NsG-354 (S-2). 
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# 

b. 

■» V 

In what follows, it will be assumed throughout that X = E n = 

Euclidean n-space. In such a space, a fuzzy set A is convex if and 
only if the sets = {x|jx A (x)> a} are convex for all a > 0. Equiv- 
alently, A is convex if and only if the inequality 

^A^ x l + ^A^ X 1 ^ * ^A^ X E^ 


holds for all x^, x^ in E n and all X. in the interval [0,1]. 


We are now ready to define what is meant by a shadow of a 
fuzzy set. Thus, let and H be, respectively, a point and a hyper- 
plane in E n . Then, a point-shadow of A on H is a fuzzy set S(A) in 
H whose membership function (x) is defined as follows: Eet L 

be a line passing through p^, with L intersecting H at a point h. 
Then, 


H S ( A ) = Su P P A (x) 

xeL 

P S (A) (x) = 0 ’ X ^ H 


( 2 ) 


Note that we use the suggestive term "point - shadow" to describe this 
fuzzy set because it bears resemblance to the shadow thrown by a cloud 
A on a plane H, with Pq acting as a point source of light. 

The transformation S which takes A into S(A) will be referred 
to as point-projection of A on H with respect to Pq- In the special 
case where Pq is a point at infinity and the lines L are orthogonal to 
H, we shall refer to S(A) and S as orthogonal shadow and orthogonal 
projection, respectively. For example, if H is the coordinate plane 
H = {x|x^ = 0}, x = (x^, . . . , x^) then the orthogonal shadow of A on 
H is characterized by the membership function 
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( 3 ) 


^(A) ^ x 2’ **•' x n ^ = Su P Pa^ X 1’ •••’ x n ) * xeH 


= 0 , x?H 

In the sequel, we shall frequently use the terms shadow and projection 
without the adjectives "point" or "orthogonal, " relying on the context 
to indicate the specific meaning in which these terms should be under- 
stood. 


We proceed to establish several basic properties of shadows of 
fuzzy sets. Most of these properties are immediate consequences of 
the defining relation (2). 

Homogeneity . Let kA denote a fuzzy set whose membership function 
is given by 

^kA (x) = k P A (x) (4) 

where k is a constant, 0 < k < 1. Then clearly 

S(kA) = kS(A) (5) 

Monotonicity . This property is expressed by the relation 

aCb =^s(A)Cs(B) (6) 

and is an immediate consequence of 

^ x [M- A (x) < H b (x)] Sup H A (x) < Sup p fi (x) 

L L 
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Distr ibutivity. For any fuzzy sets A and B, we have 


S(A U B) = S(A) \J S(B) (7) 

which implies that S is distributive with respect to U. This follows 
at once from the identity 

Sup Max(p A (x), P B (x)) = Max /sup P A (x), Sup p b (x) N ‘ . (8) 

L ' L L / 

In connection with 18), it is natural to raise the question: Is 
S distributive with respect to 0 , that is, is it true that 

S(A 0 B) = S(A)fi S(B) . (9) 

In this case, the corresponding relation in terms of membership functions 
reads 


Sup Min (P A (x), P B (x)) = Min ( Sup p A (x), Sup p B (A) ) (10) 

L L L 

This relation is not valid for arbitrary |J^(x) and p^(x). However, 
it can be made valid by suitably restricting p A (x) and p (x) , as is 

^ A xj 

done in the case of the minimax theorem. 

Note that by combining (5) and (1), we have for any constants 
k j and in [ 0, 1 ] , 

S(kj AU k 2 B) = kjStAjU k 2 S(B) (11) 

This identity indicates that S is a linear transformation, with the 
restriction that k^, € [0,1] . Note also that S is idempotent, i. e. , 

S 2 (A) = S(S(A)) = S(A). 
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Invariance of convexity * Let A be a convex fuzzy set in E n . Then 
S(A) is a convex fuzzy set in H. 


Proof. We shall prove this assertion under the simplifying assumption 
that, for all L, Sup p^(x) is attained for some x, so that Sup can be 

L L 

replaced by Max. 

Let L^ and L^ be two lines passing through p^, and let hj 
and h^ be the points at which they intersect H. Let = Mj , 

Pg(A)^2^ = ^2’ an< ^ X 1 anc * x 2 k e points on L^ and > respec- 
tively, at which p^(x) attains the values and • Then, by the 

convexity of A, for any point x on the line segment [x^, x^], 
x = X.x^ + (1 - X>)x 2 , 0 < \ < 1 , 


P A ( x) > Min(M 1 , M 2 ) 


Now, consider a line L passing through p^ and x. This line will 
intersect H at the point h = \hj + (1 - \)h 2 9 anc i by the definition of 
S(A) we can write 


P s(A ) (h) > ^A (x) - Min < M l> M 2 ) , he [h r h 2 ] (12) 

which shows that S(A) is convex. 

Equality of convex fuzzy sets . A useful property of shadows of convex 
sets is expressed by: If A and B are convex sets and S(A) = S(B) for 
all Pq (and a fixed H), then A = B. 

Proof . It will be sufficient to show that if A 4 B, then there exists a 
Pq such that S(A) 4 S(B) . 

Assuming that A 4- B, let Xq be a point at which P^Xq) i- 
e. g. , for concreteness, P^( x q) = a > p^(xq) = (3 . Since B is a convex 
set, the set = (x|pg(x) > (3} is a convex set and hence there exists 
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a hyperplane F supporting and passing through • In relation to 
F, we have J JL g(x) < (3 for all x on F and on the side of F not con- 
taining Tp. 

Now let Pq be an arbitrarily chosen point on F , and let L be 
a line passing through p^ and Xq. At the intersection, h, of this line 
with H (which may be at infinity), we have 

- P 

but on the other hand fi^(h) > a since M-^(xq) = a. Consequently, 

P A (h) + jx B (h). Q. E. D. 

In the case of orthogonal shadows, the statement of the property 
in question becomes: If A and B are convex sets and S(A) = S(B) for 
all H, then A = B- More generally, if A and B are not necessarily 
convex, then the conclusion A = B would be replaced by the weaker 
equality conv A = conv B, when conv A denotes the convex hull of A, 
that is, the smallest convex fuzzy set containing A. 
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